77 Posts About Java™

Java logo

Builders and Manipulators

QR code

Builders and Manipulators

  • Palo Alto, CA
  • comments

Here is a simple principle for naming methods in OOP, which I’m trying to follow in my code: it’s a verb if it manipulates, it’s a noun if it builds. That’s it. Nothing in between. Methods like saveFile() or getTitle() don’t fit and must be renamed and refactored. Moreover, methods that “manipulate” must always return void, for example print() or save(). Let me explain.

The Night Of (2016) by Richard Price et al.
The Night Of (2016) by Richard Price et al.

First, I have to say that this idea is very similar to the one suggested by Bertrand Meyer in his book Object Oriented Software Construction, where he proposes we divide an object’s methods into two sharply separated categories: queries and commands.

The idea behind this principle is rather philosophical. Let’s start with builders, which are supposed to create or find an object and then return it. Suppose I have a store of books and I ask it to give me a book by name:

interface Bookshelf {
  Book find(String title);
}

It’s obviously a “builder” (or a “query” in Meyer’s terms). I ask for a book and it’s given to me. The problem, though, is with the name of the method. It’s called “find,” which implies that I know how the book will be dealt with. It will be found.

However, this is not how we should treat our objects. We must not tell them how to do the job we want them to do. Instead, we must let them decide whether the book will be found, constructed, or maybe taken from a memory cache. When we query, we have to say what result we are looking for and let the object make the decision about the way this result is going to be built. A much more appropriate name for this method would be book():

interface Bookshelf {
  Book book(String title);
}

The rule of thumb is: a builder is always a noun. If the method returns something, it has to be a noun. Preferably its name should explain what the method returns. If it’s a book, name it book(). If it’s a file, call the method file(), etc. Here are a few good builder examples:

interface Foo {
  float speed(Actor actor);
  Money salary(User user);
  File database();
  Date deadline(Project project, User user);
}

Here, on the contrary, are a few examples of badly named builders:

interface Foo {
  float calculateSpeed(Actor actor);
  Money getSalary(User user);
  File openDatabase();
  Date readDeadline(Project project, User user);
}

There is no place for a verb in a builder’s name!

It’s not only about the name, by the way. A builder, since its name doesn’t contain a verb, should not do any modifications to the encapsulated entities. It may only create or find something and return it. Just like a pure function, it must not have any side-effects.

Next, there are “manipulators” (or “commands” in Meyer’s terms). They do some work for us, modifying the entities, which the object encapsulates. They are the opposite to builders, because they actually make changes to the world abstracted by the object. For example, we ask the Bookshelf to add a new book to itself:

interface Bookshelf {
  void add(Book book);
}

The method adds the book to the storage. How exactly the storage will be modified, we don’t know. But we know that since the name of the method is a verb, there will be modifications.

Also, manipulators must not return anything. It’s always void that we see as the type of their response. This is needed mostly in order to separate the imperative part of the code from the declarative part. We either receive objects or tell them what to do. We must not mix those activities in one method.

The purpose of these rules is to make the code simpler. If you follow them, and all your builders only return objects and your manipulators only modify the world, the entire design will become easier to understand. Methods will be smaller and their names shorter.

Of course, very often you will have a hard time finding those names. From time to time you will want to return something from a manipulator or make your builder make some changes, say to the cache. Try to resist this temptation and stay with the principle: a method is either a builder or a manipulator, nothing in the middle. The examples above are rather primitive, the code in real life is much more complicated. But that’s what the principle is going to help us with—making the code simpler.

I’m also aware of the noun/verb principle, which suggests always naming classes as nouns and their methods as verbs. I believe it’s a wrong idea, since it doesn’t differentiate builders from manipulators and encourages us to always think in terms of imperative instructions. I believe that OOP must be much more about declarative composition of objects, even if we have to sometimes get them from other objects instead of instantiating them via constructors. That’s why we do need builders in most situations and we also have to see an obvious difference between them and the other methods, manipulators.

You can find a more detailed discussion of this problem in Elegant Objects, Volume 1, Section 2.4.

Here is a simple principle for naming methods in OOP, which I’m trying to follow in my code: it’s a verb if it manipulates, it’s a noun if it builds. That’s it. Nothing in between. Methods like saveFile() or getTitle() don’t fit and must be renamed and refactored. Moreover, methods that “manipulate” must always return void, for example print() or save(). Let me explain.

The Night Of (2016) by Richard Price et al.
The Night Of (2016) by Richard Price et al.

First, I have to say that this idea is very similar to the one suggested by Bertrand Meyer in his book Object Oriented Software Construction, where he proposes we divide an object’s methods into two sharply separated categories: queries and commands.

The idea behind this principle is rather philosophical. Let’s start with builders, which are supposed to create or find an object and then return it. Suppose I have a store of books and I ask it to give me a book by name:

interface Bookshelf {
  Book find(String title);
}

It’s obviously a “builder” (or a “query” in Meyer’s terms). I ask for a book and it’s given to me. The problem, though, is with the name of the method. It’s called “find,” which implies that I know how the book will be dealt with. It will be found.

However, this is not how we should treat our objects. We must not tell them how to do the job we want them to do. Instead, we must let them decide whether the book will be found, constructed, or maybe taken from a memory cache. When we query, we have to say what result we are looking for and let the object make the decision about the way this result is going to be built. A much more appropriate name for this method would be book():

interface Bookshelf {
  Book book(String title);
}

The rule of thumb is: a builder is always a noun. If the method returns something, it has to be a noun. Preferably its name should explain what the method returns. If it’s a book, name it book(). If it’s a file, call the method file(), etc. Here are a few good builder examples:

interface Foo {
  float speed(Actor actor);
  Money salary(User user);
  File database();
  Date deadline(Project project, User user);
}

Here, on the contrary, are a few examples of badly named builders:

interface Foo {
  float calculateSpeed(Actor actor);
  Money getSalary(User user);
  File openDatabase();
  Date readDeadline(Project project, User user);
}

There is no place for a verb in a builder’s name!

It’s not only about the name, by the way. A builder, since its name doesn’t contain a verb, should not do any modifications to the encapsulated entities. It may only create or find something and return it. Just like a pure function, it must not have any side-effects.

Next, there are “manipulators” (or “commands” in Meyer’s terms). They do some work for us, modifying the entities, which the object encapsulates. They are the opposite to builders, because they actually make changes to the world abstracted by the object. For example, we ask the Bookshelf to add a new book to itself:

interface Bookshelf {
  void add(Book book);
}

The method adds the book to the storage. How exactly the storage will be modified, we don’t know. But we know that since the name of the method is a verb, there will be modifications.

Also, manipulators must not return anything. It’s always void that we see as the type of their response. This is needed mostly in order to separate the imperative part of the code from the declarative part. We either receive objects or tell them what to do. We must not mix those activities in one method.

The purpose of these rules is to make the code simpler. If you follow them, and all your builders only return objects and your manipulators only modify the world, the entire design will become easier to understand. Methods will be smaller and their names shorter.

Of course, very often you will have a hard time finding those names. From time to time you will want to return something from a manipulator or make your builder make some changes, say to the cache. Try to resist this temptation and stay with the principle: a method is either a builder or a manipulator, nothing in the middle. The examples above are rather primitive, the code in real life is much more complicated. But that’s what the principle is going to help us with—making the code simpler.

I’m also aware of the noun/verb principle, which suggests always naming classes as nouns and their methods as verbs. I believe it’s a wrong idea, since it doesn’t differentiate builders from manipulators and encourages us to always think in terms of imperative instructions. I believe that OOP must be much more about declarative composition of objects, even if we have to sometimes get them from other objects instead of instantiating them via constructors. That’s why we do need builders in most situations and we also have to see an obvious difference between them and the other methods, manipulators.

You can find a more detailed discussion of this problem in Elegant Objects, Volume 1, Section 2.4.

© Yegor Bugayenko 2014–2018

How I Test My Java Classes for Thread-Safety

QR code

How I Test My Java Classes for Thread-Safety

  • Moscow, Russia
  • comments

I touched on this problem in one of my recent webinars, now it’s time to explain it in writing. Thread-safety is an important quality of classes in languages/platforms like Java, where we frequently share objects between threads. The issues caused by lack of thread-safety are very difficult to debug, since they are sporadic and almost impossible to reproduce on purpose. How do you test your objects to make sure they are thread-safe? Here is how I’m doing it.

Scent of a Woman (1992) by Martin Brest
Scent of a Woman (1992) by Martin Brest

Let us say there is a simple in-memory bookshelf:

class Books {
  final Map<Integer, String> map =
    new ConcurrentHashMap<>();
  int add(String title) {
    final Integer next = this.map.size() + 1;
    this.map.put(next, title);
    return next;
  }
  String title(int id) {
    return this.map.get(id);
  }
}

First, we put a book there and the bookshelf returns its ID. Then we can read the title of the book by its ID:

Books books = new Books();
String title = "Elegant Objects";
int id = books.add(title);
assert books.title(id).equals(title);

The class seems to be thread-safe, since we are using the thread-safe ConcurrentHashMap instead of a more primitive and non-thread-safe HashMap, right? Let’s try to test it:

class BooksTest {
  @Test
  public void addsAndRetrieves() {
    Books books = new Books();
    String title = "Elegant Objects";
    int id = books.add(title);
    assert books.title(id).equals(title);
  }
}

The test passes, but it’s just a one-thread test. Let’s try to do the same manipulation from a few parallel threads (I’m using Hamcrest):

class BooksTest {
  @Test
  public void addsAndRetrieves() {
    Books books = new Books();
    int threads = 10;
    ExecutorService service =
      Executors.newFixedThreadPool(threads);
    Collection<Future<Integer>> futures =
      new ArrayList<>(threads);
    for (int t = 0; t < threads; ++t) {
      final String title = String.format("Book #%d", t);
      futures.add(service.submit(() -> books.add(title)));
    }
    Set<Integer> ids = new HashSet<>();
    for (Future<Integer> f : futures) {
      ids.add(f.get());
    }
    assertThat(ids.size(), equalTo(threads));
  }
}

First, I create a pool of threads via Executors. Then I submit ten objects of type Callable via submit(). Each of them will add a new unique book to the bookshelf. All of them will be executed, in some unpredictable order, by some of those ten threads from the pool.

Then I fetch the results of their executors through the list of objects of type Future. Finally, I calculate the amount of unique book IDs created. If the number is 10, there were no conflicts. I’m using the Set collection in order to make sure the list of IDs contains only unique elements.

The test passes on my laptop. However, it’s not strong enough. The problem here is that it’s not really testing the Books from multiple parallel threads. The time that passes between our calls to submit() is large enough to finish the execution of books.add(). That’s why in reality only one thread will run at the same time. We can check that by modifying the code a bit:

AtomicBoolean running = new AtomicBoolean();
AtomicInteger overlaps = new AtomicInteger();
Collection<Future<Integer>> futures =
  new ArrayList<>(threads);
for (int t = 0; t < threads; ++t) {
  final String title = String.format("Book #%d", t);
  futures.add(
    service.submit(
      () -> {
        if (running.get()) {
          overlaps.incrementAndGet();
        }
        running.set(true);
        int id = books.add(title);
        running.set(false);
        return id;
      }
    )
  );
}
assertThat(overlaps.get(), greaterThan(0));

With this code I’m trying to see how often threads overlap each other and do something in parallel. This never happens and overlaps is equal to zero. Thus our test is not really testing anything yet. It just adds ten books to the bookshelf one by one. If I increase the amount of threads to 1000, they start to overlap sometimes. But we want them to overlap even when there’s a small number of them. To solve that we need to use CountDownLatch:

CountDownLatch latch = new CountDownLatch(1);
AtomicBoolean running = new AtomicBoolean();
AtomicInteger overlaps = new AtomicInteger();
Collection<Future<Integer>> futures =
  new ArrayList<>(threads);
for (int t = 0; t < threads; ++t) {
  final String title = String.format("Book #%d", t);
  futures.add(
    service.submit(
      () -> {
        latch.await();
        if (running.get()) {
          overlaps.incrementAndGet();
        }
        running.set(true);
        int id = books.add(title);
        running.set(false);
        return id;
      }
    )
  );
}
latch.countDown();
Set<Integer> ids = new HashSet<>();
for (Future<Integer> f : futures) {
  ids.add(f.get());
}
assertThat(overlaps.get(), greaterThan(0));

Now each thread, before touching the books, waits for the permission given by latch. When we submit them all via submit() they stay on hold and wait. Then we release the latch with countDown() and they all start to go, simultaneously. Now, on my laptop, overlaps is equal to 3-5 even when threads is 10.

And that last assertThat() crashes now! I’m not getting 10 book IDs, as I was before. It’s 7-9, but never 10. The class, apparently, is not thread-safe!

But before we fix the class, let’s make our test simpler. Let’s use RunInThreads from Cactoos, which does exactly the same as we’ve done above, but under the hood:

class BooksTest {
  @Test
  public void addsAndRetrieves() {
    Books books = new Books();
    MatcherAssert.assertThat(
      t -> {
        String title = String.format(
          "Book #%d", t.getAndIncrement()
        );
        int id = books.add(title);
        return books.title(id).equals(title);
      },
      new RunsInThreads<>(new AtomicInteger(), 10)
    );
  }
}
badge

The first argument of assertThat() is an instance of Func (a functional interface), accepting an AtomicInteger (the first argument of RunsInThreads) and returning Boolean. This function will be executed on 10 parallel thread, using the same latch-based approach as demonstrated above.

This RunInThreads seems to be compact and convenient, I’m using it in a few projects already.

By the way, in order to make Books thread-safe we just need to add synchronized to its method add(). Or maybe you can suggest a better solution?

P.S. I learned all this from Java Concurrency in Practice by Goetz et al.

I touched on this problem in one of my recent webinars, now it’s time to explain it in writing. Thread-safety is an important quality of classes in languages/platforms like Java, where we frequently share objects between threads. The issues caused by lack of thread-safety are very difficult to debug, since they are sporadic and almost impossible to reproduce on purpose. How do you test your objects to make sure they are thread-safe? Here is how I’m doing it.

Scent of a Woman (1992) by Martin Brest
Scent of a Woman (1992) by Martin Brest

Let us say there is a simple in-memory bookshelf:

class Books {
  final Map<Integer, String> map =
    new ConcurrentHashMap<>();
  int add(String title) {
    final Integer next = this.map.size() + 1;
    this.map.put(next, title);
    return next;
  }
  String title(int id) {
    return this.map.get(id);
  }
}

First, we put a book there and the bookshelf returns its ID. Then we can read the title of the book by its ID:

Books books = new Books();
String title = "Elegant Objects";
int id = books.add(title);
assert books.title(id).equals(title);

The class seems to be thread-safe, since we are using the thread-safe ConcurrentHashMap instead of a more primitive and non-thread-safe HashMap, right? Let’s try to test it:

class BooksTest {
  @Test
  public void addsAndRetrieves() {
    Books books = new Books();
    String title = "Elegant Objects";
    int id = books.add(title);
    assert books.title(id).equals(title);
  }
}

The test passes, but it’s just a one-thread test. Let’s try to do the same manipulation from a few parallel threads (I’m using Hamcrest):

class BooksTest {
  @Test
  public void addsAndRetrieves() {
    Books books = new Books();
    int threads = 10;
    ExecutorService service =
      Executors.newFixedThreadPool(threads);
    Collection<Future<Integer>> futures =
      new ArrayList<>(threads);
    for (int t = 0; t < threads; ++t) {
      final String title = String.format("Book #%d", t);
      futures.add(service.submit(() -> books.add(title)));
    }
    Set<Integer> ids = new HashSet<>();
    for (Future<Integer> f : futures) {
      ids.add(f.get());
    }
    assertThat(ids.size(), equalTo(threads));
  }
}

First, I create a pool of threads via Executors. Then I submit ten objects of type Callable via submit(). Each of them will add a new unique book to the bookshelf. All of them will be executed, in some unpredictable order, by some of those ten threads from the pool.

Then I fetch the results of their executors through the list of objects of type Future. Finally, I calculate the amount of unique book IDs created. If the number is 10, there were no conflicts. I’m using the Set collection in order to make sure the list of IDs contains only unique elements.

The test passes on my laptop. However, it’s not strong enough. The problem here is that it’s not really testing the Books from multiple parallel threads. The time that passes between our calls to submit() is large enough to finish the execution of books.add(). That’s why in reality only one thread will run at the same time. We can check that by modifying the code a bit:

AtomicBoolean running = new AtomicBoolean();
AtomicInteger overlaps = new AtomicInteger();
Collection<Future<Integer>> futures =
  new ArrayList<>(threads);
for (int t = 0; t < threads; ++t) {
  final String title = String.format("Book #%d", t);
  futures.add(
    service.submit(
      () -> {
        if (running.get()) {
          overlaps.incrementAndGet();
        }
        running.set(true);
        int id = books.add(title);
        running.set(false);
        return id;
      }
    )
  );
}
assertThat(overlaps.get(), greaterThan(0));

With this code I’m trying to see how often threads overlap each other and do something in parallel. This never happens and overlaps is equal to zero. Thus our test is not really testing anything yet. It just adds ten books to the bookshelf one by one. If I increase the amount of threads to 1000, they start to overlap sometimes. But we want them to overlap even when there’s a small number of them. To solve that we need to use CountDownLatch:

CountDownLatch latch = new CountDownLatch(1);
AtomicBoolean running = new AtomicBoolean();
AtomicInteger overlaps = new AtomicInteger();
Collection<Future<Integer>> futures =
  new ArrayList<>(threads);
for (int t = 0; t < threads; ++t) {
  final String title = String.format("Book #%d", t);
  futures.add(
    service.submit(
      () -> {
        latch.await();
        if (running.get()) {
          overlaps.incrementAndGet();
        }
        running.set(true);
        int id = books.add(title);
        running.set(false);
        return id;
      }
    )
  );
}
latch.countDown();
Set<Integer> ids = new HashSet<>();
for (Future<Integer> f : futures) {
  ids.add(f.get());
}
assertThat(overlaps.get(), greaterThan(0));

Now each thread, before touching the books, waits for the permission given by latch. When we submit them all via submit() they stay on hold and wait. Then we release the latch with countDown() and they all start to go, simultaneously. Now, on my laptop, overlaps is equal to 3-5 even when threads is 10.

And that last assertThat() crashes now! I’m not getting 10 book IDs, as I was before. It’s 7-9, but never 10. The class, apparently, is not thread-safe!

But before we fix the class, let’s make our test simpler. Let’s use RunInThreads from Cactoos, which does exactly the same as we’ve done above, but under the hood:

class BooksTest {
  @Test
  public void addsAndRetrieves() {
    Books books = new Books();
    MatcherAssert.assertThat(
      t -> {
        String title = String.format(
          "Book #%d", t.getAndIncrement()
        );
        int id = books.add(title);
        return books.title(id).equals(title);
      },
      new RunsInThreads<>(new AtomicInteger(), 10)
    );
  }
}
badge

The first argument of assertThat() is an instance of Func (a functional interface), accepting an AtomicInteger (the first argument of RunsInThreads) and returning Boolean. This function will be executed on 10 parallel thread, using the same latch-based approach as demonstrated above.

This RunInThreads seems to be compact and convenient, I’m using it in a few projects already.

By the way, in order to make Books thread-safe we just need to add synchronized to its method add(). Or maybe you can suggest a better solution?

P.S. I learned all this from Java Concurrency in Practice by Goetz et al.

© Yegor Bugayenko 2014–2018

Fluent Interfaces Are Bad for Maintainability

QR code

Fluent Interfaces Are Bad for Maintainability

  • Moscow, Russia
  • comments

Fluent interface, first coined as a term by Martin Fowler, is a very convenient way of communicating with objects in OOP. It makes their facades easier to use and understand. However, it ruins their internal design, making them more difficult to maintain. A few words were said about that by Marco Pivetta in his blog post Fluent Interfaces are Evil; now I will add my few cents.

Donnie Brasco (1997) by Mike Newell
Donnie Brasco (1997) by Mike Newell

Let’s take my own library jcabi-http, which I created a few years ago, when I thought that fluent interfaces were a good thing. Here is how you use the library to make an HTTP request and validate its output:

String html = new JdkRequest("https://www.google.com")
  .method("GET")
  .fetch()
  .as(RestResponse.class)
  .assertStatus(200)
  .body();

This convenient method chaining makes the code short and obvious, right? Yes, it does, on the surface. But the internal design of the library’s classes, including JdkRequest, which is the one you see, is very far from being elegant. The biggest problem is that they are rather big and it’s difficult impossible to extend them without making them even bigger.

For example, right now JdkRequest has the methods method(), fetch(), and a few others. What happens when new functionality is required? The only way to add to it would be to make the class bigger, by adding new methods, which is how we jeopardize its maintainability. Here, for example, we added multipartBody() and here we added timeout().

I always feel scared when I get a new feature request in jcabi-http. I understand that it most probably means adding new methods to Request, Response, and other already bloated interfaces and classes.

I actually tried to do something in the library in order to solve this problem but it wasn’t easy. Look at this .as(RestResponse.class) method call. What it does is decorate a Response with RestResponse, in order to make it method-richer. I just didn’t want to make Response contain 50+ methods, like many other libraries do. Here is what it does (this is pseudo-code):

class Response {
  RestResponse as() {
    return new RestResponse(this);
  }
  // Seven methods
}
class RestResponse implements Response {
  private final Response origin;
  // Original seven methods from Response
  // Additional 14 methods
}

As you see, instead of adding all possible methods to Response I placed them in supplementary decorators RestResponse, JsonResponse, XmlResponse, and others. It helps, but in order to write these decorators with the central object of type Response we have to use that “ugly” method as(), which depends heavily on Reflection and type casting.

In other words, fluent interfaces mean large classes or some ugly workarounds. I mentioned this problem earlier, when I wrote about Streams API and the interface Stream, which is perfectly fluent. There are 43 methods!

That is the biggest problem with fluent interfaces—they force objects to be huge.

Fluent interfaces are perfect for their users, since all methods are in one place and the amount of classes is very small. It is easy to use them, especially with code auto-completion in most IDEs. They also make client code more readable, since “fluent” constructs look similar to plain English (aka DSL).

That is all true! However, the damage they cause to object design is the price, which is too high.

What is the alternative?

I would recommend you use decorators and smart objects instead. Here is how I would design jcabi-http, if I could do it now:

String html = new BodyOfResponse(
  new ResponseAssertStatus(
    new RequestWithMethod(
      new JdkRequest("https://www.google.com"),
      "GET"
    ),
    200
  )
).toString();

This is the same code as in the first snippet above, but it is much more object-oriented. The obvious problem with this code, of course, is that the IDE won’t be able to auto-complete almost anything. Also, we will have to remember many of the names of the classes. And the construct looks rather difficult to read for those who are used to fluent interfaces. In addition, it’s very far away from the DSL idea.

But here is the list of benefits. First, each object is small, very cohesive and they are all loosely coupled—which are obvious merits in OOP. Second, adding new functionality to the library is as easy as creating a new class; no need to touch existing classes. Third, unit testing is simplified, since classes are small. Fourth, all classes can be immutable, which is also an obvious merit in OOP.

Thus, there seems to be a conflict between usefulness and maintainability. Fluent interfaces are good for users, but bad for library developers. Small objects are good for developers, but difficult to understand and use.

It seems to be so, but only if you are used to large classes and procedural programming. To me, a large amount of small classes seems to be an advantage, not a drawback. Libraries that are clear, simple, and readable inside are much easier to use, even when I don’t know exactly which classes out there are the most suitable for me. Even without the code-auto-complete I can figure it out myself, because the code is clean.

Also, I very often find myself interested in extending existing functionality either inside my code base or via a pull request to the library. I am much more interested to do that if I know that the changes I introduce are isolated and easy to test.

Thus, no fluent interfaces anymore from me, only objects and decorators.

Fluent interface, first coined as a term by Martin Fowler, is a very convenient way of communicating with objects in OOP. It makes their facades easier to use and understand. However, it ruins their internal design, making them more difficult to maintain. A few words were said about that by Marco Pivetta in his blog post Fluent Interfaces are Evil; now I will add my few cents.

Donnie Brasco (1997) by Mike Newell
Donnie Brasco (1997) by Mike Newell

Let’s take my own library jcabi-http, which I created a few years ago, when I thought that fluent interfaces were a good thing. Here is how you use the library to make an HTTP request and validate its output:

String html = new JdkRequest("https://www.google.com")
  .method("GET")
  .fetch()
  .as(RestResponse.class)
  .assertStatus(200)
  .body();

This convenient method chaining makes the code short and obvious, right? Yes, it does, on the surface. But the internal design of the library’s classes, including JdkRequest, which is the one you see, is very far from being elegant. The biggest problem is that they are rather big and it’s difficult impossible to extend them without making them even bigger.

For example, right now JdkRequest has the methods method(), fetch(), and a few others. What happens when new functionality is required? The only way to add to it would be to make the class bigger, by adding new methods, which is how we jeopardize its maintainability. Here, for example, we added multipartBody() and here we added timeout().

I always feel scared when I get a new feature request in jcabi-http. I understand that it most probably means adding new methods to Request, Response, and other already bloated interfaces and classes.

I actually tried to do something in the library in order to solve this problem but it wasn’t easy. Look at this .as(RestResponse.class) method call. What it does is decorate a Response with RestResponse, in order to make it method-richer. I just didn’t want to make Response contain 50+ methods, like many other libraries do. Here is what it does (this is pseudo-code):

class Response {
  RestResponse as() {
    return new RestResponse(this);
  }
  // Seven methods
}
class RestResponse implements Response {
  private final Response origin;
  // Original seven methods from Response
  // Additional 14 methods
}

As you see, instead of adding all possible methods to Response I placed them in supplementary decorators RestResponse, JsonResponse, XmlResponse, and others. It helps, but in order to write these decorators with the central object of type Response we have to use that “ugly” method as(), which depends heavily on Reflection and type casting.

In other words, fluent interfaces mean large classes or some ugly workarounds. I mentioned this problem earlier, when I wrote about Streams API and the interface Stream, which is perfectly fluent. There are 43 methods!

That is the biggest problem with fluent interfaces—they force objects to be huge.

Fluent interfaces are perfect for their users, since all methods are in one place and the amount of classes is very small. It is easy to use them, especially with code auto-completion in most IDEs. They also make client code more readable, since “fluent” constructs look similar to plain English (aka DSL).

That is all true! However, the damage they cause to object design is the price, which is too high.

What is the alternative?

I would recommend you use decorators and smart objects instead. Here is how I would design jcabi-http, if I could do it now:

String html = new BodyOfResponse(
  new ResponseAssertStatus(
    new RequestWithMethod(
      new JdkRequest("https://www.google.com"),
      "GET"
    ),
    200
  )
).toString();

This is the same code as in the first snippet above, but it is much more object-oriented. The obvious problem with this code, of course, is that the IDE won’t be able to auto-complete almost anything. Also, we will have to remember many of the names of the classes. And the construct looks rather difficult to read for those who are used to fluent interfaces. In addition, it’s very far away from the DSL idea.

But here is the list of benefits. First, each object is small, very cohesive and they are all loosely coupled—which are obvious merits in OOP. Second, adding new functionality to the library is as easy as creating a new class; no need to touch existing classes. Third, unit testing is simplified, since classes are small. Fourth, all classes can be immutable, which is also an obvious merit in OOP.

Thus, there seems to be a conflict between usefulness and maintainability. Fluent interfaces are good for users, but bad for library developers. Small objects are good for developers, but difficult to understand and use.

It seems to be so, but only if you are used to large classes and procedural programming. To me, a large amount of small classes seems to be an advantage, not a drawback. Libraries that are clear, simple, and readable inside are much easier to use, even when I don’t know exactly which classes out there are the most suitable for me. Even without the code-auto-complete I can figure it out myself, because the code is clean.

Also, I very often find myself interested in extending existing functionality either inside my code base or via a pull request to the library. I am much more interested to do that if I know that the changes I introduce are isolated and easy to test.

Thus, no fluent interfaces anymore from me, only objects and decorators.

© Yegor Bugayenko 2014–2018

Don't Parse, Use Parsing Objects

QR code

Don't Parse, Use Parsing Objects

  • Moscow, Russia
  • comments

The traditional way of integrating object-oriented back-end with an external system is through data transfer objects, which are serialized into JSON before going out and deserialized when coming back. This way is as much popular as it is wrong. The serialization part should be replaced by printers, which I explained earlier. Here is my take on deserialization, which should be done by—guess what—objects.

La science des rêves (2006) by Michel Gondry
La science des rêves (2006) by Michel Gondry

Say there is a back-end entry point, which is supposed to register a new book in the library, arriving in JSON:

{
  "title": "Object Thinking",
  "isbn: "0735619654",
  "author: "David West"
}

Also, there is an object of class Library, which expects an object of type Book to be given to its method register():

class Library {
  public void register(Book book) {
    // Create a new record in the database
  }
}

Say also, type Book has a simple method isbn():

interface Book {
  String isbn();
}

Now, here is the HTTP entry point (I’m using Takes and Cactoos), which is accepting a POST multipart/form-data request and registering the book in the library:

public class TkUpload implements Take {
  private final Library library;
  @Override
  public Response act(Request req) {
    String body = new RqPrint(
      new RqMtSmart(new RqMtBase(req)).single("book")
    ).printBody();
    JsonObject json = Json.createReader(
      new InputStreamOf(body)
    ).readObject();
    Book book = new BookDTO();
    book.setIsbn(json.getString("isbn"));
    library.register(book);
  }
}

What is wrong with this? Well, a few things.

First, it’s not reusable. If we were to need something similar in a different place, we would have to write this HTTP processing and JSON parsing again.

Second, error handling and validation are not reusable either. If we add it to the method above, we will have to copy it everywhere. Of course, the DTO may encapsulate it, but that’s not what DTOs are usually for.

Third, the code above is rather procedural and has a lot of temporal coupling.

A better design would be to hide this parsing inside a new class JsonBook:

class JsonBook implements Book {
  private final String json;
  JsonBook(String body) {
    this.json = body;
  }
  @Override
  public String isbn() {
    return Json.createReader(
      new InputStreamOf(body)
    ).readObject().getString("isbn");
  }
}

Then, the RESTful entry point will look like this:

public class TkUpload implements Take {
  private final Library library;
  @Override
  public Response act(Request req) {
    library.register(
      new JsonBook(
        new RqPrint(
          new RqMtSmart(
            new RqMtBase(req)
          ).single("book")
        ).printBody()
      )
    );
  }
}

Isn’t that more elegant?

Here are some examples from my projects: RqUser from zerocracy/farm and RqUser from yegor256/jare.

As you can see from the examples above, sometimes we can’t use implements because some primitives in Java are not interfaces but final classes: String is a “perfect” example. That’s why I have to do this:

class RqUser implements Scalar<String> {
  @Override
  public String value() {
    // Parsing happens here and returns String
  }
}

But aside from that, these examples perfectly demonstrate the principle of “parsing objects” suggested above.

The traditional way of integrating object-oriented back-end with an external system is through data transfer objects, which are serialized into JSON before going out and deserialized when coming back. This way is as much popular as it is wrong. The serialization part should be replaced by printers, which I explained earlier. Here is my take on deserialization, which should be done by—guess what—objects.

La science des rêves (2006) by Michel Gondry
La science des rêves (2006) by Michel Gondry

Say there is a back-end entry point, which is supposed to register a new book in the library, arriving in JSON:

{
  "title": "Object Thinking",
  "isbn: "0735619654",
  "author: "David West"
}

Also, there is an object of class Library, which expects an object of type Book to be given to its method register():

class Library {
  public void register(Book book) {
    // Create a new record in the database
  }
}

Say also, type Book has a simple method isbn():

interface Book {
  String isbn();
}

Now, here is the HTTP entry point (I’m using Takes and Cactoos), which is accepting a POST multipart/form-data request and registering the book in the library:

public class TkUpload implements Take {
  private final Library library;
  @Override
  public Response act(Request req) {
    String body = new RqPrint(
      new RqMtSmart(new RqMtBase(req)).single("book")
    ).printBody();
    JsonObject json = Json.createReader(
      new InputStreamOf(body)
    ).readObject();
    Book book = new BookDTO();
    book.setIsbn(json.getString("isbn"));
    library.register(book);
  }
}

What is wrong with this? Well, a few things.

First, it’s not reusable. If we were to need something similar in a different place, we would have to write this HTTP processing and JSON parsing again.

Second, error handling and validation are not reusable either. If we add it to the method above, we will have to copy it everywhere. Of course, the DTO may encapsulate it, but that’s not what DTOs are usually for.

Third, the code above is rather procedural and has a lot of temporal coupling.

A better design would be to hide this parsing inside a new class JsonBook:

class JsonBook implements Book {
  private final String json;
  JsonBook(String body) {
    this.json = body;
  }
  @Override
  public String isbn() {
    return Json.createReader(
      new InputStreamOf(body)
    ).readObject().getString("isbn");
  }
}

Then, the RESTful entry point will look like this:

public class TkUpload implements Take {
  private final Library library;
  @Override
  public Response act(Request req) {
    library.register(
      new JsonBook(
        new RqPrint(
          new RqMtSmart(
            new RqMtBase(req)
          ).single("book")
        ).printBody()
      )
    );
  }
}

Isn’t that more elegant?

Here are some examples from my projects: RqUser from zerocracy/farm and RqUser from yegor256/jare.

As you can see from the examples above, sometimes we can’t use implements because some primitives in Java are not interfaces but final classes: String is a “perfect” example. That’s why I have to do this:

class RqUser implements Scalar<String> {
  @Override
  public String value() {
    // Parsing happens here and returns String
  }
}

But aside from that, these examples perfectly demonstrate the principle of “parsing objects” suggested above.

© Yegor Bugayenko 2014–2018

Alan Kay Was Wrong About Him Being Wrong

QR code

Alan Kay Was Wrong About Him Being Wrong

  • Moscow, Russia
  • comments

From time to time someone asks me what I think about what Alan Kay, the father of OOP, the designer of Smalltalk, the first object-oriented language, said in 1998 about OOP. He literally said that the very term “object” was misleading and a more appropriate one would be “messaging.” Here is what I think.

Rain Man (1988) by Barry Levinson
Rain Man (1988) by Barry Levinson

I believe that there are two orthogonal means of interaction between objects: messaging and composition. Let’s say, we have a point and a canvas:

Point p = new Point(x, y);
Canvas canvas = new Canvas();

This is how messaging would look:

p.printTo(canvas);

The problem with messaging is that it keeps objects on the same level of abstraction. They communicate as equal and independent “modules,” sending data messages to each other. Even though they look object-oriented, the entire communication pattern is very procedural. We try to encapsulate as much as we can inside a single object, however inevitably still having to expose a lot of its data in order to be able to “connect” it with other objects.

We turn objects into “little computers,” as some books refer to them. They expect data to come in, they process the data, and return back some new data. The maintainability problem is not really solved with this approach—we still have to deal with a lot of data, remembering its semantic outside of the objects. In other words, there is no true encapsulation.

On the other hand, this is how composition would look instead:

Point p2 = new PrintedOn(p, canvas);

Every time we need objects to communicate we create a bigger object that encapsulates more primitive ones, letting them interact inside. Of course, the data will also go from object to object, but that will happen inside a bigger object. We can even make the encapsulator and the encapsulated “friends,” as I suggested before, to make that interaction more transparent and avoid data exposure through getters or even printers.

Let me quote Alan Kay again:

The key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be.

It seems to me that he means modules, which are not objects. These are different things. Modules are elements of the architecture, while objects are elements of the design. These are two different levels. At the level of architecture we obviously need messages and Kay’s statement is perfectly correct. However, at the level of design we need composable structures, to increase maintainability and messaging is not what can help us achieve this goal.

Thus, I believe Alan Kay was right when he invented objects, called them objects, and gave their programming style the “object-oriented” title.

From time to time someone asks me what I think about what Alan Kay, the father of OOP, the designer of Smalltalk, the first object-oriented language, said in 1998 about OOP. He literally said that the very term “object” was misleading and a more appropriate one would be “messaging.” Here is what I think.

Rain Man (1988) by Barry Levinson
Rain Man (1988) by Barry Levinson

I believe that there are two orthogonal means of interaction between objects: messaging and composition. Let’s say, we have a point and a canvas:

Point p = new Point(x, y);
Canvas canvas = new Canvas();

This is how messaging would look:

p.printTo(canvas);

The problem with messaging is that it keeps objects on the same level of abstraction. They communicate as equal and independent “modules,” sending data messages to each other. Even though they look object-oriented, the entire communication pattern is very procedural. We try to encapsulate as much as we can inside a single object, however inevitably still having to expose a lot of its data in order to be able to “connect” it with other objects.

We turn objects into “little computers,” as some books refer to them. They expect data to come in, they process the data, and return back some new data. The maintainability problem is not really solved with this approach—we still have to deal with a lot of data, remembering its semantic outside of the objects. In other words, there is no true encapsulation.

On the other hand, this is how composition would look instead:

Point p2 = new PrintedOn(p, canvas);

Every time we need objects to communicate we create a bigger object that encapsulates more primitive ones, letting them interact inside. Of course, the data will also go from object to object, but that will happen inside a bigger object. We can even make the encapsulator and the encapsulated “friends,” as I suggested before, to make that interaction more transparent and avoid data exposure through getters or even printers.

Let me quote Alan Kay again:

The key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be.

It seems to me that he means modules, which are not objects. These are different things. Modules are elements of the architecture, while objects are elements of the design. These are two different levels. At the level of architecture we obviously need messages and Kay’s statement is perfectly correct. However, at the level of design we need composable structures, to increase maintainability and messaging is not what can help us achieve this goal.

Thus, I believe Alan Kay was right when he invented objects, called them objects, and gave their programming style the “object-oriented” title.

© Yegor Bugayenko 2014–2018

Constructors or Static Factory Methods?

QR code

Constructors or Static Factory Methods?

  • Odessa, Ukraine
  • comments

I believe Joshua Bloch said it first in his very good book “Effective Java”: static factory methods are the preferred way to instantiate objects compared with constructors. I disagree. Not only because I believe that static methods are pure evil, but mostly because in this particular case they pretend to be good and make us think that we have to love them.

Extract (2009) by Mike Judge
Extract (2009) by Mike Judge

Let’s analyze the reasoning and see why it’s wrong, from an object-oriented point of view.

This is a class with one primary and two secondary constructors:

class Color {
  private final int hex;
  Color(String rgb) {
    this(Integer.parseInt(rgb, 16));
  }
  Color(int red, int green, int blue) {
    this(red << 16 + green << 8 + blue);
  }
  Color(int h) {
    this.hex = h;
  }
}

This is a similar class with three static factory methods:

class Color {
  private final int hex;
  static Color makeFromRGB(String rgb) {
    return new Color(Integer.parseInt(rgb, 16));
  }
  static Color makeFromPalette(int red, int green, int blue) {
    return new Color(red << 16 + green << 8 + blue);
  }
  static Color makeFromHex(int h) {
    return new Color(h);
  }
  private Color(int h) {
    this.hex = h;
  }
}

Which one do you like better?

According to Joshua Bloch, there are three basic advantages to using static factory methods instead of constructors (there are actually four, but the fourth one is not applicable to Java anymore):

  • They have names.
  • They can cache.
  • They can subtype.

I believe that all three make perfect sense … if the design is wrong. They are good excuses for workarounds. Let’s take them one by one.

They Have Names

This is how you make a red tomato color object with a constructor:

Color tomato = new Color(255, 99, 71);

This is how you do it with a static factory method:

Color tomato = Color.makeFromPalette(255, 99, 71);

It seems that makeFromPalette() is semantically richer than just new Color(), right? Well, yes. Who knows what those three numbers mean if we just pass them to the constructor. But the word “palette” helps us figure everything out immediately.

True.

However, the right solution would be to use polymorphism and encapsulation, to decompose the problem into a few semantically rich classes:

interface Color {
}
class HexColor implements Color {
  private final int hex;
  HexColor(int h) {
    this.hex = h;
  }
}
class RGBColor implements Color {
  private final Color origin;
  RGBColor(int red, int green, int blue) {
    this.origin = new HexColor(
      red << 16 + green << 8 + blue
    );
  }
}

Now, we use the right constructor of the right class:

Color tomato = new RGBColor(255, 99, 71);

See, Joshua?

They Can Cache

Let’s say I need a red tomato color in multiple places in the application:

Color tomato = new Color(255, 99, 71);
// ... sometime later
Color red = new Color(255, 99, 71);

Two objects will be created, which is obviously inefficient, since they are identical. It would be better to keep the first instance somewhere in memory and return it when the second call arrives. Static factory methods make it possible to solve this very problem:

Color tomato = Color.makeFromPalette(255, 99, 71);
// ... sometime later
Color red = Color.makeFromPalette(255, 99, 71);

Then somewhere inside the Color we keep a private static Map with all the objects already instantiated:

class Color {
  private static final Map<Integer, Color> CACHE =
    new HashMap<>();
  private final int hex;
  static Color makeFromPalette(int red, int green, int blue) {
    final int hex = red << 16 + green << 8 + blue;
    return Color.CACHE.computeIfAbsent(
      hex, h -> new Color(h)
    );
  }
  private Color(int h) {
    return new Color(h);
  }
}

It is very effective performance-wise. With a small object like our Color the problem may not be so obvious, but when objects are bigger, their instantiation and garbage collection may waste a lot of time.

True.

However, there is an object-oriented way to solve this problem. We just introduce a new class Palette, which becomes a store of colors:

class Palette {
  private final Map<Integer, Color> colors =
    new HashMap<>();
  Color take(int red, int green, int blue) {
    final int hex = red << 16 + green << 8 + blue;
    return this.computerIfAbsent(
      hex, h -> new Color(h)
    );
  }
}

Now, we make an instance of Palette once and ask it to return a color to us every time we need it:

Color tomato = palette.take(255, 99, 71);
// Later we will get the same instance:
Color red = palette.take(255, 99, 71);

See, Joshua, no static methods, no static attributes.

They Can Subtype

Let’s say our class Color has a method lighter(), which is supposed to shift the color to the next available lighter one:

class Color {
  protected final int hex;
  Color(int h) {
    this.hex = h;
  }
  public Color lighter() {
    return new Color(hex + 0x111);
  }
}

However, sometimes it’s more desirable to pick the next lighter color through a set of available Pantone colors:

class PantoneColor extends Color {
  private final PantoneName pantone;
  PantoneColor(String name) {
    this(new PantoneName(name));
  }
  PantoneColor(PantoneName name) {
    this.pantone = name;
  }
  @Override
  public Color lighter() {
    return new PantoneColor(this.pantone.up());
  }
}

Then, we create a static factory method, which will decide which Color implementation is the most suitable for us:

class Color {
  private final String code;
  static Color make(int h) {
    if (h == 0xBF1932) {
      return new PantoneColor("19-1664 TPX");
    }
    return new RGBColor(h);
  }
}

If the true red color is requested, we return an instance of PantoneColor. In all other cases it’s just a standard RGBColor. The decision is made by the static factory method. This is how we will call it:

Color color = Color.make(0xBF1932);

It would not be possible to do the same “forking” with a constructor, since it can only return the class it is declared in. A static method has all the necessary freedom to return any subtype of Color.

True.

However, in an object-oriented world we can and must do it all differently. First, we would make Color an interface:

interface Color {
  Color lighter();
}

Next, we would move this decision making process to its own class Colors, just like we did in the previous example:

class Colors {
  Color make(int h) {
    if (h == 0xBF1932) {
      return new PantoneColor("19-1664-TPX");
    }
    return new RGBColor(h);
  }
}

And we would use an instance of class Colors instead of a static faсtory method inside Color:

colors.make(0xBF1932);

However, this is still not really an object-oriented way of thinking, because we’re taking the decision-making away from the object it belongs to. Either through a static factory method make() or a new class Colors—it doesn’t really matter how—we tear our objects into two pieces. The first piece is the object itself and the second one is the decision making algorithm that stays somewhere else.

A much more object-oriented design would be to put the logic into an object of class PantoneColor which would decorate the original RGBColor:

class PantoneColor {
  private final Color origin;
  PantoneColor(Color color) {
    this.origin = color;
  }
  @Override
  public Color lighter() {
    final Color next;
    if (this.origin.hex() == 0xBF1932) {
      next = new RGBColor(0xD12631);
    } else {
      next = this.origin.lighter();
    }
    return new PantoneColor(next);
  }
)

Then, we make an instance of RGBColor and decorate it with PantoneColor:

Color red = new PantoneColor(
  new RGBColor(0xBF1932)
);

We ask red to return a lighter color and it returns the one from the Pantone palette, not the one that is merely lighter in RGB coordinates:

Color lighter = red.lighter(); // 0xD12631

Of course, this example is rather primitive and needs further improvement if we really want it to be applicable to all Pantone colors, but I hope you get the idea. The logic must stay inside the class, not somewhere outside, not in static factory methods or even in some other supplementary class. I’m talking about the logic that belongs to this particular class, of course. If it’s something related to the management of class instances, then there can be containers and stores, just like in the previous example above.

To summarize, I would strongly recommend you never use static methods, especially when they are going to replace object constructors. Giving birth to an object through its constructor is the most “sacred” moment in any object-oriented software, don’t miss the beauty of it.

I believe Joshua Bloch said it first in his very good book “Effective Java”: static factory methods are the preferred way to instantiate objects compared with constructors. I disagree. Not only because I believe that static methods are pure evil, but mostly because in this particular case they pretend to be good and make us think that we have to love them.

Extract (2009) by Mike Judge
Extract (2009) by Mike Judge

Let’s analyze the reasoning and see why it’s wrong, from an object-oriented point of view.

This is a class with one primary and two secondary constructors:

class Color {
  private final int hex;
  Color(String rgb) {
    this(Integer.parseInt(rgb, 16));
  }
  Color(int red, int green, int blue) {
    this(red << 16 + green << 8 + blue);
  }
  Color(int h) {
    this.hex = h;
  }
}

This is a similar class with three static factory methods:

class Color {
  private final int hex;
  static Color makeFromRGB(String rgb) {
    return new Color(Integer.parseInt(rgb, 16));
  }
  static Color makeFromPalette(int red, int green, int blue) {
    return new Color(red << 16 + green << 8 + blue);
  }
  static Color makeFromHex(int h) {
    return new Color(h);
  }
  private Color(int h) {
    this.hex = h;
  }
}

Which one do you like better?

According to Joshua Bloch, there are three basic advantages to using static factory methods instead of constructors (there are actually four, but the fourth one is not applicable to Java anymore):

  • They have names.
  • They can cache.
  • They can subtype.

I believe that all three make perfect sense … if the design is wrong. They are good excuses for workarounds. Let’s take them one by one.

They Have Names

This is how you make a red tomato color object with a constructor:

Color tomato = new Color(255, 99, 71);

This is how you do it with a static factory method:

Color tomato = Color.makeFromPalette(255, 99, 71);

It seems that makeFromPalette() is semantically richer than just new Color(), right? Well, yes. Who knows what those three numbers mean if we just pass them to the constructor. But the word “palette” helps us figure everything out immediately.

True.

However, the right solution would be to use polymorphism and encapsulation, to decompose the problem into a few semantically rich classes:

interface Color {
}
class HexColor implements Color {
  private final int hex;
  HexColor(int h) {
    this.hex = h;
  }
}
class RGBColor implements Color {
  private final Color origin;
  RGBColor(int red, int green, int blue) {
    this.origin = new HexColor(
      red << 16 + green << 8 + blue
    );
  }
}

Now, we use the right constructor of the right class:

Color tomato = new RGBColor(255, 99, 71);

See, Joshua?

They Can Cache

Let’s say I need a red tomato color in multiple places in the application:

Color tomato = new Color(255, 99, 71);
// ... sometime later
Color red = new Color(255, 99, 71);

Two objects will be created, which is obviously inefficient, since they are identical. It would be better to keep the first instance somewhere in memory and return it when the second call arrives. Static factory methods make it possible to solve this very problem:

Color tomato = Color.makeFromPalette(255, 99, 71);
// ... sometime later
Color red = Color.makeFromPalette(255, 99, 71);

Then somewhere inside the Color we keep a private static Map with all the objects already instantiated:

class Color {
  private static final Map<Integer, Color> CACHE =
    new HashMap<>();
  private final int hex;
  static Color makeFromPalette(int red, int green, int blue) {
    final int hex = red << 16 + green << 8 + blue;
    return Color.CACHE.computeIfAbsent(
      hex, h -> new Color(h)
    );
  }
  private Color(int h) {
    return new Color(h);
  }
}

It is very effective performance-wise. With a small object like our Color the problem may not be so obvious, but when objects are bigger, their instantiation and garbage collection may waste a lot of time.

True.

However, there is an object-oriented way to solve this problem. We just introduce a new class Palette, which becomes a store of colors:

class Palette {
  private final Map<Integer, Color> colors =
    new HashMap<>();
  Color take(int red, int green, int blue) {
    final int hex = red << 16 + green << 8 + blue;
    return this.computerIfAbsent(
      hex, h -> new Color(h)
    );
  }
}

Now, we make an instance of Palette once and ask it to return a color to us every time we need it:

Color tomato = palette.take(255, 99, 71);
// Later we will get the same instance:
Color red = palette.take(255, 99, 71);

See, Joshua, no static methods, no static attributes.

They Can Subtype

Let’s say our class Color has a method lighter(), which is supposed to shift the color to the next available lighter one:

class Color {
  protected final int hex;
  Color(int h) {
    this.hex = h;
  }
  public Color lighter() {
    return new Color(hex + 0x111);
  }
}

However, sometimes it’s more desirable to pick the next lighter color through a set of available Pantone colors:

class PantoneColor extends Color {
  private final PantoneName pantone;
  PantoneColor(String name) {
    this(new PantoneName(name));
  }
  PantoneColor(PantoneName name) {
    this.pantone = name;
  }
  @Override
  public Color lighter() {
    return new PantoneColor(this.pantone.up());
  }
}

Then, we create a static factory method, which will decide which Color implementation is the most suitable for us:

class Color {
  private final String code;
  static Color make(int h) {
    if (h == 0xBF1932) {
      return new PantoneColor("19-1664 TPX");
    }
    return new RGBColor(h);
  }
}

If the true red color is requested, we return an instance of PantoneColor. In all other cases it’s just a standard RGBColor. The decision is made by the static factory method. This is how we will call it:

Color color = Color.make(0xBF1932);

It would not be possible to do the same “forking” with a constructor, since it can only return the class it is declared in. A static method has all the necessary freedom to return any subtype of Color.

True.

However, in an object-oriented world we can and must do it all differently. First, we would make Color an interface:

interface Color {
  Color lighter();
}

Next, we would move this decision making process to its own class Colors, just like we did in the previous example:

class Colors {
  Color make(int h) {
    if (h == 0xBF1932) {
      return new PantoneColor("19-1664-TPX");
    }
    return new RGBColor(h);
  }
}

And we would use an instance of class Colors instead of a static faсtory method inside Color:

colors.make(0xBF1932);

However, this is still not really an object-oriented way of thinking, because we’re taking the decision-making away from the object it belongs to. Either through a static factory method make() or a new class Colors—it doesn’t really matter how—we tear our objects into two pieces. The first piece is the object itself and the second one is the decision making algorithm that stays somewhere else.

A much more object-oriented design would be to put the logic into an object of class PantoneColor which would decorate the original RGBColor:

class PantoneColor {
  private final Color origin;
  PantoneColor(Color color) {
    this.origin = color;
  }
  @Override
  public Color lighter() {
    final Color next;
    if (this.origin.hex() == 0xBF1932) {
      next = new RGBColor(0xD12631);
    } else {
      next = this.origin.lighter();
    }
    return new PantoneColor(next);
  }
)

Then, we make an instance of RGBColor and decorate it with PantoneColor:

Color red = new PantoneColor(
  new RGBColor(0xBF1932)
);

We ask red to return a lighter color and it returns the one from the Pantone palette, not the one that is merely lighter in RGB coordinates:

Color lighter = red.lighter(); // 0xD12631

Of course, this example is rather primitive and needs further improvement if we really want it to be applicable to all Pantone colors, but I hope you get the idea. The logic must stay inside the class, not somewhere outside, not in static factory methods or even in some other supplementary class. I’m talking about the logic that belongs to this particular class, of course. If it’s something related to the management of class instances, then there can be containers and stores, just like in the previous example above.

To summarize, I would strongly recommend you never use static methods, especially when they are going to replace object constructors. Giving birth to an object through its constructor is the most “sacred” moment in any object-oriented software, don’t miss the beauty of it.

© Yegor Bugayenko 2014–2018

Five Features to Make Java Even Better

QR code

Five Features to Make Java Even Better

  • Odessa, Ukraine
  • comments

I stumbled upon this proposal by Brian Goetz for data classes in Java, and immediately realized that I too have a few ideas about how to make Java better as a language. I actually have many of them, but this is a short list of the five most important.

Idiocracy (2006) by Mike Judge
Idiocracy (2006) by Mike Judge

Global Variables. There are Singletons in Java, which, as we all know, are nothing but global variables. Wouldn’t it be great to enable global variables in Java and get rid of Singletons. PHP, JavaScript, Ruby and many other languages have them, why doesn’t Java? Look at this code:

class User {
  private static User INSTANCE;
  private User() {}
  public static User getInstance() {
    synchronized (User.INSTANCE) {
      if (User.INSTANCE == null) {
        User.INSTANCE = new User();
      }
    }
    return User.INSTANCE;
  }
  public String getName() {
    // return user's name
  }
}

Then, to access it we have to use:

String name = User.getInstance().getName();

This is a Singleton. See how verbose it is? We can simply replace it with a global variable (global is the keyword I’m suggesting we use):

global User user;

And then:

user.getName();

Much less code to write, and way easier to read!

Global Functions and Namespaces

To group static methods together we create utility classes, where we have to define private constructors to prevent their instantiation. Also, we have to remember which particular utility class a static method is in. It’s just extra hassle. I’m suggesting we add global functions to Java and optional “namespaces” to group them. Take a look at this utility class:

class TextUtils {
  private TextUtils() {}
  public static String trim(String text) {
    if (text == null) {
      return "";
    }
    return text.trim();
  }
}

Now look at this global function with a namespace:

namespace TextUtils {
  String trim(String text) {
    if (text == null) {
      return "";
    }
    return text.trim();
  }
}

My point is that since we are already using classes as collections of functions, let’s make it more convenient. In some applications we won’t even need namespaces, just global functions, like in C and C++.

Full Access to Private Attributes and Methods

In order to access a private attribute or a method of an object from outside we have to use the Reflection API. It’s not particularly difficult, but it does take a few lines of code, which are not so easy to read and understand:

class Point {
  private int x;
  private int y;
}
Point point = new Point();
Field field = point.getClass().getDeclaredField("x");
field.setAccessible(true);
int x = (int) field.get(point);

I’m suggesting we allow any object to access any of the attributes and methods of another object:

Point point = new Point();
int x = point.x;

Of course, if they are private, the compiler will issue a warning. At compile time you simply ignore the warning and move on. If you really care about encapsulation, pay attention to the warning and do something else. But in most cases programmers will ignore it, since they would happily use the Reflection API anyway.

NULL by Default

It would be convenient to let us call constructors and methods with an incomplete set of arguments. The arguments we don’t provide will be set to null by default. Also, when a method has to return something, but there is no return statement, Java should return null. This is almost exactly how it works in PHP, Ruby, and many other languages. I believe it would be a convenient feature for Java monkeys developers too.

We won’t need to define so many methods when some of the arguments are optional. Method overloading is very verbose and difficult to understand. Instead, we should have one method with a long list of arguments. Some of them will be provided by the caller, others will be set to null. The method will decide what to do, for example:

void save(File file, String encoding) {
 if (encoding == null) {
   encoding = "UTF-8";
 }
}

Then we just call either save(f) or save(f, "UTF-16"). The method will understand what we mean. We can also make it even more convenient, like it’s done in Ruby, providing method arguments by names:

save(file: f, encoding: "UTF-16");

Also, when there is nothing to return, the method must return null by default. Writing return null is just a waste of a code line and doesn’t really improve readability. Take a look:

String load(File file) {
 if (file.exists()) {
   return read_the_content();
 }
}

It’s obvious from this code that if the file exists, the method loads and returns its content. If not, it returns null, which will be a good indicator for the caller that something is not right and the content of the file is not available.

Getters and Setters

I think it’s only obvious that we need this feature: every private attribute must automatically have a setter and a getter. There should be no need to create them, Java will provide them out-of-the-box, just like Kotlin and Ruby do. What is the point of having an attribute if there are no getters and setters to read it and to modify it, right?

With this new feature we’ll no longer need the help of Lombok, or IntelliJ IDEA.


Maybe I should turn my ideas into official proposals to JCP. What do you think?

I stumbled upon this proposal by Brian Goetz for data classes in Java, and immediately realized that I too have a few ideas about how to make Java better as a language. I actually have many of them, but this is a short list of the five most important.

Idiocracy (2006) by Mike Judge
Idiocracy (2006) by Mike Judge

Global Variables. There are Singletons in Java, which, as we all know, are nothing but global variables. Wouldn’t it be great to enable global variables in Java and get rid of Singletons. PHP, JavaScript, Ruby and many other languages have them, why doesn’t Java? Look at this code:

class User {
  private static User INSTANCE;
  private User() {}
  public static User getInstance() {
    synchronized (User.INSTANCE) {
      if (User.INSTANCE == null) {
        User.INSTANCE = new User();
      }
    }
    return User.INSTANCE;
  }
  public String getName() {
    // return user's name
  }
}

Then, to access it we have to use:

String name = User.getInstance().getName();

This is a Singleton. See how verbose it is? We can simply replace it with a global variable (global is the keyword I’m suggesting we use):

global User user;

And then:

user.getName();

Much less code to write, and way easier to read!

Global Functions and Namespaces

To group static methods together we create utility classes, where we have to define private constructors to prevent their instantiation. Also, we have to remember which particular utility class a static method is in. It’s just extra hassle. I’m suggesting we add global functions to Java and optional “namespaces” to group them. Take a look at this utility class:

class TextUtils {
  private TextUtils() {}
  public static String trim(String text) {
    if (text == null) {
      return "";
    }
    return text.trim();
  }
}

Now look at this global function with a namespace:

namespace TextUtils {
  String trim(String text) {
    if (text == null) {
      return "";
    }
    return text.trim();
  }
}

My point is that since we are already using classes as collections of functions, let’s make it more convenient. In some applications we won’t even need namespaces, just global functions, like in C and C++.

Full Access to Private Attributes and Methods

In order to access a private attribute or a method of an object from outside we have to use the Reflection API. It’s not particularly difficult, but it does take a few lines of code, which are not so easy to read and understand:

class Point {
  private int x;
  private int y;
}
Point point = new Point();
Field field = point.getClass().getDeclaredField("x");
field.setAccessible(true);
int x = (int) field.get(point);

I’m suggesting we allow any object to access any of the attributes and methods of another object:

Point point = new Point();
int x = point.x;

Of course, if they are private, the compiler will issue a warning. At compile time you simply ignore the warning and move on. If you really care about encapsulation, pay attention to the warning and do something else. But in most cases programmers will ignore it, since they would happily use the Reflection API anyway.

NULL by Default

It would be convenient to let us call constructors and methods with an incomplete set of arguments. The arguments we don’t provide will be set to null by default. Also, when a method has to return something, but there is no return statement, Java should return null. This is almost exactly how it works in PHP, Ruby, and many other languages. I believe it would be a convenient feature for Java monkeys developers too.

We won’t need to define so many methods when some of the arguments are optional. Method overloading is very verbose and difficult to understand. Instead, we should have one method with a long list of arguments. Some of them will be provided by the caller, others will be set to null. The method will decide what to do, for example:

void save(File file, String encoding) {
 if (encoding == null) {
   encoding = "UTF-8";
 }
}

Then we just call either save(f) or save(f, "UTF-16"). The method will understand what we mean. We can also make it even more convenient, like it’s done in Ruby, providing method arguments by names:

save(file: f, encoding: "UTF-16");

Also, when there is nothing to return, the method must return null by default. Writing return null is just a waste of a code line and doesn’t really improve readability. Take a look:

String load(File file) {
 if (file.exists()) {
   return read_the_content();
 }
}

It’s obvious from this code that if the file exists, the method loads and returns its content. If not, it returns null, which will be a good indicator for the caller that something is not right and the content of the file is not available.

Getters and Setters

I think it’s only obvious that we need this feature: every private attribute must automatically have a setter and a getter. There should be no need to create them, Java will provide them out-of-the-box, just like Kotlin and Ruby do. What is the point of having an attribute if there are no getters and setters to read it and to modify it, right?

With this new feature we’ll no longer need the help of Lombok, or IntelliJ IDEA.


Maybe I should turn my ideas into official proposals to JCP. What do you think?

© Yegor Bugayenko 2014–2018

Lazy Loading and Caching via Sticky Cactoos Primitives

QR code

Lazy Loading and Caching via Sticky Cactoos Primitives

  • Odessa, Ukraine
  • comments

You obviously know what lazy loading is, right? And you no doubt know about caching. To my knowledge, there is no elegant way in Java to implement either of them. Here is what I found out for myself with the help of Cactoos primitives.

Reality (2012) by Matteo Garrone
Reality (2012) by Matteo Garrone

Let’s say we need an object that will encrypt some text. Speaking in a more object-oriented way, it will encapsulate the text and become its encrypted form. Here is how we will use it (let’s create tests first):

interface Encrypted {
  String asString() throws IOException;
}
Encrypted enc = new EncryptedX("Hello, world!");
System.out.println(enc.asString());

Now let’s implement it, in a very primitive way, with one primary constructor. The encryption mechanism will just add +1 to each byte in the incoming data, and will assume that the encryption won’t break anything (a very stupid assumption, but for the sake of this example it will work):

class Encrypted1 implements Encrypted {
  private final String text;
  Encrypted1(String txt) {
    this.text = txt;
  }
  @Override
  public String asString() {
    final byte[] in = this.text.getBytes();
    final byte[] out = new byte[in.length];
    for (int i = 0; i < in.length; ++i) {
      out[i] = (byte) (in[i] + 1);
    }
    return new String(out);
  }
}

Looks correct so far? I tested it and it works. If the input is "Hello, world!", the output will be "Ifmmp-!xpsme\"".

Next, let’s say that we want our class to accept an InputStream as well as a String. We want to call it like this, for example:

Encrypted enc = new Encrypted2(
  new FileInputStream("/tmp/hello.txt")
);
System.out.println(enc.toString());

Here is the most obvious implementation, with two primary constructors (again, the implementation is primitive, but works):

class Encrypted2 implements Encrypted {
  private final String text;
  Encrypted2(InputStream input) throws IOException {
    ByteArrayOutputStream baos =
      new ByteArrayOutputStream();
    while (true) {
      int one = input.read();
      if (one < 0) {
        break;
      }
      baos.write(one);
    }
    this.text = new String(baos.toByteArray());
  }
  Encrypted2(String txt) {
    this.text = txt;
  }
  // asString() is exactly the same as in Encrypted1
}

Technically it works, but stream reading is right inside the constructor, which is bad practice. Primary constructors must not do anything but attribute assignments, while secondary ones may only create new objects.

Let’s try to refactor and introduce lazy loading:

class Encrypted3 implements Encrypted {
  private String text;
  private final InputStream input;
  Encrypted3(InputStream stream) {
    this.text = null;
    this.input = stream;
  }
  Encrypted3(String txt) {
    this.text = txt;
    this.input = null;
  }
  @Override
  public String asString() throws IOException {
    if (this.text == null) {
      ByteArrayOutputStream baos =
        new ByteArrayOutputStream();
      while (true) {
        int one = input.read();
        if (one < 0) {
          break;
        }
        baos.write(one);
      }
      this.text = new String(baos.toByteArray());
    }
    final byte[] in = this.text.getBytes();
    final byte[] out = new byte[in.length];
    for (int i = 0; i < in.length; ++i) {
      out[i] = (byte) (in[i] + 1);
    }
    return new String(out);
  }
}

Works great, but looks ugly. The ugliest part is these two lines of course:

this.text = null;
this.input = null;

They make the object mutable and they’re using NULL. It’s ugly, trust me. Unfortunately, lazy loading and NULL references always come together in classic examples. However there is a better way to implement it. Let’s refactor our class, this time using Scalar from Cactoos:

class Encrypted4 implements Encrypted {
  private final IoCheckedScalar<String> text;
  Encrypted4(InputStream stream) {
    this(
      () -> {
        ByteArrayOutputStream baos =
          new ByteArrayOutputStream();
        while (true) {
          int one = stream.read();
          if (one < 0) {
            break;
          }
          baos.write(one);
        }
        return new String(baos.toByteArray());
      }
    );
  }
  Encrypted4(String txt) {
    this(() -> txt);
  }
  Encrypted4(Scalar<String> source) {
    this.text = new IoCheckedScalar<>(source);
  }
  @Override
  public String asString() throws IOException {
    final byte[] in = this.text.value().getBytes();
    final byte[] out = new byte[in.length];
    for (int i = 0; i < in.length; ++i) {
      out[i] = (byte) (in[i] + 1);
    }
    return new String(out);
  }

Now it looks way better. First of all, there is only one primary constructor and two secondary ones. Second, the object is immutable. Third, there is still a lot of room for improvement: we can add more constructors which will accept other sources of data, for example File or a byte array.

In a nutshell, the attribute that is supposed to be loaded in a “lazy” way is represented inside an object as a “function” (lambda expression in Java 8). Until we touch that attribute, it’s not loaded. Once we need to work with it, the function gets executed and we have the result.

There is one problem with this code though. It will read the input stream every time we call asString(), which will obviously not work, since only the first time will the stream have the data. On every subsequent call the stream will simply be empty. Thus, we need to make sure that this.text.value() executes the encapsulated Scalar only once. All later calls must return the previously calculated value. So we need to cache it. Here is how:

class Encrypted5 implements Encrypted {
  private final IoCheckedScalar<String> text;
  // same as above in Encrypted4
  Encrypted5(Scalar<String> source) {
    this.text = new IoCheckedScalar<>(
      new StickyScalar<>(source)
    );
  }
  // same as above in Encrypted4

This StickyScalar will make sure that only the first call to its method value() will go through to the encapsulated Scalar. All other calls will receive the result of the first call.

The last problem to solve is about concurrency. The code we have above is not thread safe. If I create an instance of Encrypted5 and pass it to two threads, which call asString() simultaneously, the result will be unpredictable, simply because StickyScalar is not thread-safe. There is another primitive to help us out though, called SyncScalar:

class Encrypted5 implements Encrypted {
  private final IoCheckedScalar<String> text;
  // same as above in Encrypted4
  Encrypted5(Scalar<String> source) {
    this.text = new IoCheckedScalar<>(
      new SyncScalar<>(
        new StickyScalar<>(source)
      )
    );
  }
  // same as above in Encrypted4

Now we’re safe and the design is elegant. It includes lazy loading and caching.

I’m using this approach in many projects now and it seems convenient, clear, and object-oriented.

You obviously know what lazy loading is, right? And you no doubt know about caching. To my knowledge, there is no elegant way in Java to implement either of them. Here is what I found out for myself with the help of Cactoos primitives.

Reality (2012) by Matteo Garrone
Reality (2012) by Matteo Garrone

Let’s say we need an object that will encrypt some text. Speaking in a more object-oriented way, it will encapsulate the text and become its encrypted form. Here is how we will use it (let’s create tests first):

interface Encrypted {
  String asString() throws IOException;
}
Encrypted enc = new EncryptedX("Hello, world!");
System.out.println(enc.asString());

Now let’s implement it, in a very primitive way, with one primary constructor. The encryption mechanism will just add +1 to each byte in the incoming data, and will assume that the encryption won’t break anything (a very stupid assumption, but for the sake of this example it will work):

class Encrypted1 implements Encrypted {
  private final String text;
  Encrypted1(String txt) {
    this.text = txt;
  }
  @Override
  public String asString() {
    final byte[] in = this.text.getBytes();
    final byte[] out = new byte[in.length];
    for (int i = 0; i < in.length; ++i) {
      out[i] = (byte) (in[i] + 1);
    }
    return new String(out);
  }
}

Looks correct so far? I tested it and it works. If the input is "Hello, world!", the output will be "Ifmmp-!xpsme\"".

Next, let’s say that we want our class to accept an InputStream as well as a String. We want to call it like this, for example:

Encrypted enc = new Encrypted2(
  new FileInputStream("/tmp/hello.txt")
);
System.out.println(enc.toString());

Here is the most obvious implementation, with two primary constructors (again, the implementation is primitive, but works):

class Encrypted2 implements Encrypted {
  private final String text;
  Encrypted2(InputStream input) throws IOException {
    ByteArrayOutputStream baos =
      new ByteArrayOutputStream();
    while (true) {
      int one = input.read();
      if (one < 0) {
        break;
      }
      baos.write(one);
    }
    this.text = new String(baos.toByteArray());
  }
  Encrypted2(String txt) {
    this.text = txt;
  }
  // asString() is exactly the same as in Encrypted1
}

Technically it works, but stream reading is right inside the constructor, which is bad practice. Primary constructors must not do anything but attribute assignments, while secondary ones may only create new objects.

Let’s try to refactor and introduce lazy loading:

class Encrypted3 implements Encrypted {
  private String text;
  private final InputStream input;
  Encrypted3(InputStream stream) {
    this.text = null;
    this.input = stream;
  }
  Encrypted3(String txt) {
    this.text = txt;
    this.input = null;
  }
  @Override
  public String asString() throws IOException {
    if (this.text == null) {
      ByteArrayOutputStream baos =
        new ByteArrayOutputStream();
      while (true) {
        int one = input.read();
        if (one < 0) {
          break;
        }
        baos.write(one);
      }
      this.text = new String(baos.toByteArray());
    }
    final byte[] in = this.text.getBytes();
    final byte[] out = new byte[in.length];
    for (int i = 0; i < in.length; ++i) {
      out[i] = (byte) (in[i] + 1);
    }
    return new String(out);
  }
}

Works great, but looks ugly. The ugliest part is these two lines of course:

this.text = null;
this.input = null;

They make the object mutable and they’re using NULL. It’s ugly, trust me. Unfortunately, lazy loading and NULL references always come together in classic examples. However there is a better way to implement it. Let’s refactor our class, this time using Scalar from Cactoos:

class Encrypted4 implements Encrypted {
  private final IoCheckedScalar<String> text;
  Encrypted4(InputStream stream) {
    this(
      () -> {
        ByteArrayOutputStream baos =
          new ByteArrayOutputStream();
        while (true) {
          int one = stream.read();
          if (one < 0) {
            break;
          }
          baos.write(one);
        }
        return new String(baos.toByteArray());
      }
    );
  }
  Encrypted4(String txt) {
    this(() -> txt);
  }
  Encrypted4(Scalar<String> source) {
    this.text = new IoCheckedScalar<>(source);
  }
  @Override
  public String asString() throws IOException {
    final byte[] in = this.text.value().getBytes();
    final byte[] out = new byte[in.length];
    for (int i = 0; i < in.length; ++i) {
      out[i] = (byte) (in[i] + 1);
    }
    return new String(out);
  }

Now it looks way better. First of all, there is only one primary constructor and two secondary ones. Second, the object is immutable. Third, there is still a lot of room for improvement: we can add more constructors which will accept other sources of data, for example File or a byte array.

In a nutshell, the attribute that is supposed to be loaded in a “lazy” way is represented inside an object as a “function” (lambda expression in Java 8). Until we touch that attribute, it’s not loaded. Once we need to work with it, the function gets executed and we have the result.

There is one problem with this code though. It will read the input stream every time we call asString(), which will obviously not work, since only the first time will the stream have the data. On every subsequent call the stream will simply be empty. Thus, we need to make sure that this.text.value() executes the encapsulated Scalar only once. All later calls must return the previously calculated value. So we need to cache it. Here is how:

class Encrypted5 implements Encrypted {
  private final IoCheckedScalar<String> text;
  // same as above in Encrypted4
  Encrypted5(Scalar<String> source) {
    this.text = new IoCheckedScalar<>(
      new StickyScalar<>(source)
    );
  }
  // same as above in Encrypted4

This StickyScalar will make sure that only the first call to its method value() will go through to the encapsulated Scalar. All other calls will receive the result of the first call.

The last problem to solve is about concurrency. The code we have above is not thread safe. If I create an instance of Encrypted5 and pass it to two threads, which call asString() simultaneously, the result will be unpredictable, simply because StickyScalar is not thread-safe. There is another primitive to help us out though, called SyncScalar:

class Encrypted5 implements Encrypted {
  private final IoCheckedScalar<String> text;
  // same as above in Encrypted4
  Encrypted5(Scalar<String> source) {
    this.text = new IoCheckedScalar<>(
      new SyncScalar<>(
        new StickyScalar<>(source)
      )
    );
  }
  // same as above in Encrypted4

Now we’re safe and the design is elegant. It includes lazy loading and caching.

I’m using this approach in many projects now and it seems convenient, clear, and object-oriented.

© Yegor Bugayenko 2014–2018

Streams vs. Decorators

QR code

Streams vs. Decorators

  • Odessa, Ukraine
  • comments

The Streams API was introduced in Java 8, together with lambda expressions, just a few years ago. I, as a disciplined Java adept, tried to use this new feature in a few of my projects, for example here and here. I didn’t really like it and went back to good old decorators. Moreover, I created Cactoos, a library of decorators, to replace Guava, which is not so good in so many places.

La Haine (1995) by Mathieu Kassovitz
La Haine (1995) by Mathieu Kassovitz

Here is a primitive example. Let’s say we have a collection of measurements coming in from some data source, they are all numbers between zero and one:

Iterable<Double> probes;

Now, we need to show only the first 10 of them, ignoring zeros and ones, and re-scaling them to (0..100). Sounds like an easy task, right? There are three ways to do it: procedural, object-oriented, and the Java 8 way. Let’s start with the procedural way:

int pos = 0;
for (Double probe : probes) {
  if (probe == 0.0d || probe == 1.0d) {
    continue;
  }
  if (++pos > 10) {
    break;
  }
  System.out.printf(
    "Probe #%d: %f", pos, probe * 100.0d
  );
}

Why is this a procedural way? Because it’s imperative. Why is it imperative? Because it’s procedural. Nah, I’m kidding.

It’s imperative because we’re giving instructions to the computer about what data to put where and how to iterate through it. We’re not declaring the result, but imperatively building it. It works, but it’s not really scalable. We can’t take part of this algorithm and apply it to another use case. We can’t really modify it easily, for example to take numbers from two sources instead of one, etc. It’s procedural. Enough said. Don’t do it this way.

Now, Java 8 gives us the Streams API, which is supposed to offer a functional way to do the same. Let’s try to use it.

First, we need to create an instance of Stream, which Iterable doesn’t let us obtain directly. Then we use the stream API to do the job:

StreamSupport.stream(probes.spliterator(), false)
  .filter(p -> p == 0.0d || p == 1.0d)
  .limit(10L)
  .forEach(
    probe -> System.out.printf(
      "Probe #%d: %f", 0, probe * 100.0d
    )
  );

This will work, but will say Probe #0 for all probes, because forEach() doesn’t work with indexes. There is no such thing as forEachWithIndex() in the Stream interface as of Java 8 (and Java 9 too). Here is a workaround with an atomic counter:

AtomicInteger index = new AtomicInteger();
StreamSupport.stream(probes.spliterator(), false)
  .filter(probe -> probe != 0.0d && probe != 1.0d)
  .limit(10L)
  .forEach(
    probe -> System.out.printf(
      "Probe #%d: %f",
      index.getAndIncrement(),
      probe * 100.0d
    )
  );

“What’s wrong with that?” you may ask. First, see how easily we got into trouble when we didn’t find the right method in the Stream interface. We immediately fell off the “streaming” paradigm and got back to the good old procedural global variable (the counter). Second, we don’t really see what’s going on inside those filter(), limit(), and forEach() methods. How exactly do they work? The documentation says that this approach is “declarative” and each method in the Stream interface returns an instance of some class. What classes are they? We have no idea by just looking at this code.

These two problems are connected. The biggest issue with this streaming API is the very interface Stream—it’s huge. At the time of writing there are 43 methods. Forty three, in a single interface! This is against each and every principle of object-oriented programming, starting with SOLID and then up to more serious ones.

What is the object-oriented way to implement the same algorithm? Here is how I would do it with Cactoos, which is just a collection of primitive simple Java classes:

new And(
  new Mapped<Double, Scalar<Boolean>>(
    new Limited<Double>(
      new Filtered<Double>(
        probes,
        probe -> probe != 0.0d && probe != 1.0d
      ),
      10
    ),
    probe -> () -> {
      System.out.printf(
        "Probe #%d: %f", 0, probe * 100.0d
      );
      return true;
    }
  )
).value();

Let’s see what’s going on here. First, Filtered decorates our iterable probes to take certain items out of it. Notice that Filtered implements Iterable. Then Limited, also being an Iterable, takes only the first ten items out. Then Mapped converts each probe into an instance of Scalar<Boolean>, which does the line printing.

Finally, the instance of And goes through the list of “scalars” and ask each of them to return boolean. They print the line and return true. Since it’s true, And makes the next attempt with the next scalar. Finally, its method value() returns true.

But wait, there are no indexes. Let’s add them. In order to do that we just use another class, called AndWithIndex:

new AndWithIndex(
  new Mapped<Double, Func<Integer, Boolean>>(
    new Limited<Double>(
      new Filtered<Double>(
        probes,
        probe -> probe != 0.0d && probe != 1.0d
      ),
      10
    ),
    probe -> index -> {
      System.out.printf(
        "Probe #%d: %f", index, probe * 100.0d
      );
      return true;
    }
  )
).value();

Instead of Scalar<Boolean> we now map our probes to Func<Integer, Boolean> to let them accept the index.

The beauty of this approach is that all classes and interfaces are small and that’s why they’re very composable. To make an iterable of probes limited we decorate it with Limited; to make it filtered we decorate it with Filtered; to do something else we create a new decorator and use it. We’re not stuck to one single interface like Stream.

The bottom line is that decorators are an object-oriented instrument to modify the behavior of collections, while streams is something else which I can’t even find the name for.

P.S. By the way, this is how the same algorithm can be implemented with the help of Guava’s Iterables:

Iterable<Double> ready = Iterables.limit(
  Iterables.filter(
    probes,
    probe -> probe != 0.0d && probe != 1.0d
  ),
  10
);
int pos = 0;
for (Double probe : probes) {
  System.out.printf(
    "Probe #%d: %f", pos++, probe * 100.0d
  );
}

This is some weird combination of object-oriented and functional styles.

The Streams API was introduced in Java 8, together with lambda expressions, just a few years ago. I, as a disciplined Java adept, tried to use this new feature in a few of my projects, for example here and here. I didn’t really like it and went back to good old decorators. Moreover, I created Cactoos, a library of decorators, to replace Guava, which is not so good in so many places.

La Haine (1995) by Mathieu Kassovitz
La Haine (1995) by Mathieu Kassovitz

Here is a primitive example. Let’s say we have a collection of measurements coming in from some data source, they are all numbers between zero and one:

Iterable<Double> probes;

Now, we need to show only the first 10 of them, ignoring zeros and ones, and re-scaling them to (0..100). Sounds like an easy task, right? There are three ways to do it: procedural, object-oriented, and the Java 8 way. Let’s start with the procedural way:

int pos = 0;
for (Double probe : probes) {
  if (probe == 0.0d || probe == 1.0d) {
    continue;
  }
  if (++pos > 10) {
    break;
  }
  System.out.printf(
    "Probe #%d: %f", pos, probe * 100.0d
  );
}

Why is this a procedural way? Because it’s imperative. Why is it imperative? Because it’s procedural. Nah, I’m kidding.

It’s imperative because we’re giving instructions to the computer about what data to put where and how to iterate through it. We’re not declaring the result, but imperatively building it. It works, but it’s not really scalable. We can’t take part of this algorithm and apply it to another use case. We can’t really modify it easily, for example to take numbers from two sources instead of one, etc. It’s procedural. Enough said. Don’t do it this way.

Now, Java 8 gives us the Streams API, which is supposed to offer a functional way to do the same. Let’s try to use it.

First, we need to create an instance of Stream, which Iterable doesn’t let us obtain directly. Then we use the stream API to do the job:

StreamSupport.stream(probes.spliterator(), false)
  .filter(p -> p == 0.0d || p == 1.0d)
  .limit(10L)
  .forEach(
    probe -> System.out.printf(
      "Probe #%d: %f", 0, probe * 100.0d
    )
  );

This will work, but will say Probe #0 for all probes, because forEach() doesn’t work with indexes. There is no such thing as forEachWithIndex() in the Stream interface as of Java 8 (and Java 9 too). Here is a workaround with an atomic counter:

AtomicInteger index = new AtomicInteger();
StreamSupport.stream(probes.spliterator(), false)
  .filter(probe -> probe != 0.0d && probe != 1.0d)
  .limit(10L)
  .forEach(
    probe -> System.out.printf(
      "Probe #%d: %f",
      index.getAndIncrement(),
      probe * 100.0d
    )
  );

“What’s wrong with that?” you may ask. First, see how easily we got into trouble when we didn’t find the right method in the Stream interface. We immediately fell off the “streaming” paradigm and got back to the good old procedural global variable (the counter). Second, we don’t really see what’s going on inside those filter(), limit(), and forEach() methods. How exactly do they work? The documentation says that this approach is “declarative” and each method in the Stream interface returns an instance of some class. What classes are they? We have no idea by just looking at this code.

These two problems are connected. The biggest issue with this streaming API is the very interface Stream—it’s huge. At the time of writing there are 43 methods. Forty three, in a single interface! This is against each and every principle of object-oriented programming, starting with SOLID and then up to more serious ones.

What is the object-oriented way to implement the same algorithm? Here is how I would do it with Cactoos, which is just a collection of primitive simple Java classes:

new And(
  new Mapped<Double, Scalar<Boolean>>(
    new Limited<Double>(
      new Filtered<Double>(
        probes,
        probe -> probe != 0.0d && probe != 1.0d
      ),
      10
    ),
    probe -> () -> {
      System.out.printf(
        "Probe #%d: %f", 0, probe * 100.0d
      );
      return true;
    }
  )
).value();

Let’s see what’s going on here. First, Filtered decorates our iterable probes to take certain items out of it. Notice that Filtered implements Iterable. Then Limited, also being an Iterable, takes only the first ten items out. Then Mapped converts each probe into an instance of Scalar<Boolean>, which does the line printing.

Finally, the instance of And goes through the list of “scalars” and ask each of them to return boolean. They print the line and return true. Since it’s true, And makes the next attempt with the next scalar. Finally, its method value() returns true.

But wait, there are no indexes. Let’s add them. In order to do that we just use another class, called AndWithIndex:

new AndWithIndex(
  new Mapped<Double, Func<Integer, Boolean>>(
    new Limited<Double>(
      new Filtered<Double>(
        probes,
        probe -> probe != 0.0d && probe != 1.0d
      ),
      10
    ),
    probe -> index -> {
      System.out.printf(
        "Probe #%d: %f", index, probe * 100.0d
      );
      return true;
    }
  )
).value();

Instead of Scalar<Boolean> we now map our probes to Func<Integer, Boolean> to let them accept the index.

The beauty of this approach is that all classes and interfaces are small and that’s why they’re very composable. To make an iterable of probes limited we decorate it with Limited; to make it filtered we decorate it with Filtered; to do something else we create a new decorator and use it. We’re not stuck to one single interface like Stream.

The bottom line is that decorators are an object-oriented instrument to modify the behavior of collections, while streams is something else which I can’t even find the name for.

P.S. By the way, this is how the same algorithm can be implemented with the help of Guava’s Iterables:

Iterable<Double> ready = Iterables.limit(
  Iterables.filter(
    probes,
    probe -> probe != 0.0d && probe != 1.0d
  ),
  10
);
int pos = 0;
for (Double probe : probes) {
  System.out.printf(
    "Probe #%d: %f", pos++, probe * 100.0d
  );
}

This is some weird combination of object-oriented and functional styles.

© Yegor Bugayenko 2014–2018

Java 9: The Good, The Bad, and Private Interface Methods

QR code

Java 9: The Good, The Bad, and Private Interface Methods

  • Odessa, Ukraine
  • comments

Java 9 was released a few weeks ago. Check the release notes, they include many interesting features. However, I think that not everything is as good as Oracle and Java adepts seem to picture it. I see three trends in the Java world, which are good, bad, and ugly, respectively. Let’s start with the good one.

Birdman (2014) by Alejandro G. Iñárritu
Birdman (2014) by Alejandro G. Iñárritu

The Platform

The first trend is an obvious improvement of the platform that compiles Java, packages JARs, and runs the bytecode. It definitely becomes better with every new Java release. Here is a list of improvements Java 9 made, which are very useful, without doubt:

The platform is obviously becoming more mature. This is a good trend.

The JDK

The second trend, which I’ve observed since Java 6, shows that the JDK, which is essentially a collection of classes and interfaces designed, developed, and maintained by Oracle, gets bigger with every new release. In Java 9 they added and extended, besides others, the following:

Of course some features must be implemented in the JDK itself, like Unicode support (JEP 267), platform-specific Desktop features (JEP 272), Spin-Wait Hints (JEP 285), compact strings (JEP 254), and the process API (JEP 102). Their implementation depends on the underlying platform and has to be provided together with the JVM.

But what is HTTP 2.0 client doing in the JDK, together with JAX-RS, JPA, JAX-WS, JDBC, and many other things that, in my opinion, should stay as far away from Oracle as possible? They are not platform specific and they can be, in a much better way, designed by the open source community as independent packages. Aggregating them under one monster umbrella brand is a mistake, I believe.

I think that big corporations are only killing the software market, instead of making it better, because of the financial and political motives they expose it to. That’s exactly what is happening with JDK. Thanks to the Oracle monopoly it lacks flexibility and dynamicity in growth. In other words, we’re stuck with what Oracle and its big friends think is right.

Thus, making JDK bigger is a bad trend. Instead, I believe, Oracle would only benefit from making it smaller, delegating everything that is not platform-specific to the open source community, supporting programmers somehow and promoting open and effective standardization processes on the market.

The Language

Java was developed by James Gosling in Sun Microsystems in 1995 as an object-oriented language. There were many concerns about this claim of object-orientation and I’m also not sure that Java is more OO than it is procedural. However it is officially object-oriented.

There were many procedural features inherited by Java from C/C++, since its first version, including static methods, NULL, implementation inheritance, etc. It was not a perfect object-oriented language and it was not going to be one, as I understand it. The key idea was to create something that could be written once and ran anywhere. However the language was a big deal also, not just the JVM. It was simple and sexy.

Java 5 made a serious step forward in 2004 and improved the language by adding generics, for-each loop, varargs, and static import. However, annotations and enumerations were introduced, which helped the language to divert from the object paradigm to something completely different and procedural.

Java 7 added try-with-resource in 2011, which was a good move, in line with the OOP paradigm.

Java 8 added lambda expressions in 2014, which was a great feature, but absolutely irrelevant to OOP. Lambda and Streams API turned Java into a mix of the object, procedural, and functional paradigms. Default methods were also added to interfaces, which turned types into libraries of code. Types into libraries! It’s even worse than implementation inheritance, if you ask me.

Now Java 9 made the next “improvement” to interfaces, allowing them to have private methods. Private static methods in types! Can you believe it? What will be the next step? Attributes, in Java 10, I guess.

Also, let’s take a look at what was done to some core classes in the JDK, to understand where the language is heading. Just two examples.

Factory methods for collections (JEP 269). Instead of introducing new constructors and allowing us to do this:

List<Integer> list = new ArrayList<>(1, 2, 3);

…in Java 9 they created more static factory methods and made us do this:

List<Integer> list = List.of(1, 2, 3);

“Fewer constructors, more static methods!” seems to be the philosophy of those who introduced this JEP. Needless to say that this is completely against the very spirit of object-oriented programming. Objects must be created by constructors, not static methods, no matter what Joshua Bloch says. Static methods make the moment of operator new usage invisible for us and that’s why the code is way less maintainable—we simply don’t know exactly what class is instantiated and what the real arguments of its ctor are.

By the way, with Cactoos you can do it the right way:

List<Integer> list = new ListOf(1, 2, 3);

This is OOP.

New methods in InputStream. Three new methods were added to the already over bloated class InputStream: transferTo(), readNBytes(), and readAllBytes(). Now we are supposed to do this, when we want input stream to copy to an output stream:

input.transferTo(output);

It’s one of the most typical mistakes young OOP programmers are making: they make their interfaces big. Just because they need more functionality. I guess the interface segregation principle is part of the famous SOLID and is many years old. What’s wrong with you, Oracle? What will the next step be? In Java 10 we will also have saveToFile() and printToConsole()? How about emailToAFriend()?

This is how you would do the same with the IOUtils utility class from commons-io:

IOUtils.copy(input, output);

It’s not perfect, but it’s better. The most object-oriented way is to use objects, not utility classes and static methods. This is how it works in Cactoos:

new LengthOf(new TeeInput(input, output)).length();

This is OOP.


In my opinion, Java is getting uglier, and this is a trend. Does it mean that it’s time to quit? No! No matter how ugly you are, we will always love you Java!

Java 9 was released a few weeks ago. Check the release notes, they include many interesting features. However, I think that not everything is as good as Oracle and Java adepts seem to picture it. I see three trends in the Java world, which are good, bad, and ugly, respectively. Let’s start with the good one.

Birdman (2014) by Alejandro G. Iñárritu
Birdman (2014) by Alejandro G. Iñárritu

The Platform

The first trend is an obvious improvement of the platform that compiles Java, packages JARs, and runs the bytecode. It definitely becomes better with every new Java release. Here is a list of improvements Java 9 made, which are very useful, without doubt:

The platform is obviously becoming more mature. This is a good trend.

The JDK

The second trend, which I’ve observed since Java 6, shows that the JDK, which is essentially a collection of classes and interfaces designed, developed, and maintained by Oracle, gets bigger with every new release. In Java 9 they added and extended, besides others, the following:

Of course some features must be implemented in the JDK itself, like Unicode support (JEP 267), platform-specific Desktop features (JEP 272), Spin-Wait Hints (JEP 285), compact strings (JEP 254), and the process API (JEP 102). Their implementation depends on the underlying platform and has to be provided together with the JVM.

But what is HTTP 2.0 client doing in the JDK, together with JAX-RS, JPA, JAX-WS, JDBC, and many other things that, in my opinion, should stay as far away from Oracle as possible? They are not platform specific and they can be, in a much better way, designed by the open source community as independent packages. Aggregating them under one monster umbrella brand is a mistake, I believe.

I think that big corporations are only killing the software market, instead of making it better, because of the financial and political motives they expose it to. That’s exactly what is happening with JDK. Thanks to the Oracle monopoly it lacks flexibility and dynamicity in growth. In other words, we’re stuck with what Oracle and its big friends think is right.

Thus, making JDK bigger is a bad trend. Instead, I believe, Oracle would only benefit from making it smaller, delegating everything that is not platform-specific to the open source community, supporting programmers somehow and promoting open and effective standardization processes on the market.

The Language

Java was developed by James Gosling in Sun Microsystems in 1995 as an object-oriented language. There were many concerns about this claim of object-orientation and I’m also not sure that Java is more OO than it is procedural. However it is officially object-oriented.

There were many procedural features inherited by Java from C/C++, since its first version, including static methods, NULL, implementation inheritance, etc. It was not a perfect object-oriented language and it was not going to be one, as I understand it. The key idea was to create something that could be written once and ran anywhere. However the language was a big deal also, not just the JVM. It was simple and sexy.

Java 5 made a serious step forward in 2004 and improved the language by adding generics, for-each loop, varargs, and static import. However, annotations and enumerations were introduced, which helped the language to divert from the object paradigm to something completely different and procedural.

Java 7 added try-with-resource in 2011, which was a good move, in line with the OOP paradigm.

Java 8 added lambda expressions in 2014, which was a great feature, but absolutely irrelevant to OOP. Lambda and Streams API turned Java into a mix of the object, procedural, and functional paradigms. Default methods were also added to interfaces, which turned types into libraries of code. Types into libraries! It’s even worse than implementation inheritance, if you ask me.

Now Java 9 made the next “improvement” to interfaces, allowing them to have private methods. Private static methods in types! Can you believe it? What will be the next step? Attributes, in Java 10, I guess.

Also, let’s take a look at what was done to some core classes in the JDK, to understand where the language is heading. Just two examples.

Factory methods for collections (JEP 269). Instead of introducing new constructors and allowing us to do this:

List<Integer> list = new ArrayList<>(1, 2, 3);

…in Java 9 they created more static factory methods and made us do this:

List<Integer> list = List.of(1, 2, 3);

“Fewer constructors, more static methods!” seems to be the philosophy of those who introduced this JEP. Needless to say that this is completely against the very spirit of object-oriented programming. Objects must be created by constructors, not static methods, no matter what Joshua Bloch says. Static methods make the moment of operator new usage invisible for us and that’s why the code is way less maintainable—we simply don’t know exactly what class is instantiated and what the real arguments of its ctor are.

By the way, with Cactoos you can do it the right way:

List<Integer> list = new ListOf(1, 2, 3);

This is OOP.

New methods in InputStream. Three new methods were added to the already over bloated class InputStream: transferTo(), readNBytes(), and readAllBytes(). Now we are supposed to do this, when we want input stream to copy to an output stream:

input.transferTo(output);

It’s one of the most typical mistakes young OOP programmers are making: they make their interfaces big. Just because they need more functionality. I guess the interface segregation principle is part of the famous SOLID and is many years old. What’s wrong with you, Oracle? What will the next step be? In Java 10 we will also have saveToFile() and printToConsole()? How about emailToAFriend()?

This is how you would do the same with the IOUtils utility class from commons-io:

IOUtils.copy(input, output);

It’s not perfect, but it’s better. The most object-oriented way is to use objects, not utility classes and static methods. This is how it works in Cactoos:

new LengthOf(new TeeInput(input, output)).length();

This is OOP.


In my opinion, Java is getting uglier, and this is a trend. Does it mean that it’s time to quit? No! No matter how ugly you are, we will always love you Java!

© Yegor Bugayenko 2014–2018

RAII in Java

QR code

RAII in Java

  • Riga, Latvia
  • comments

Resource Acquisition Is Initialization (RAII) is a design idea introduced in C++ by Bjarne Stroustrup for exception-safe resource management. Thanks to garbage collection Java doesn’t have this feature, but we can implement something similar, using try-with-resources.

At Sachem Farm (1998) by John Huddles
At Sachem Farm (1998) by John Huddles

The problem RAII is solving is obvious; have a look at this code (I’m sure you know what Semaphore is and how it works in Java):

class Foo {
  private Semaphore sem = new Semaphore(5);
  void print(int x) throws Exception {
    this.sem.acquire();
    if (x > 1000) {
      throw new Exception("Too large!");
    }
    System.out.printf("x = %d", x);
    this.sem.release();
  }
}

The code is rather primitive and doesn’t do anything useful, but you most probably get the idea: the method print(), if being called from multiple parallel threads, will allow only five of them to print in parallel. Sometimes it will not allow some of them to print and will throw an exception if x is bigger than 1000.

The problem with this code is—resource leakage. Each print() call with x larger than 1000 will take one permit from the semaphore and won’t return it. In five calls with exceptions the semaphore will be empty and all other threads won’t print anything.

What is the solution? Here it is:

class Foo {
  private Semaphore sem = new Semaphore(5);
  void print(int x) throws Exception {
    this.sem.acquire();
    if (x > 1000) {
      this.sem.release();
      throw new Exception("Too large!");
    }
    System.out.printf("x = %d", x);
    this.sem.release();
  }
}

We must release the permit before we throw the exception.

However, there is another problem that shows up: code duplication. We release the permit in two places. If we add more throw instructions we will also have to add more sem.release() calls.

A very elegant solution was introduced in C++ and is called RAII. This is how it would look in Java:

class Permit {
  private Semaphore sem;
  Permit(Semaphore s) {
    this.sem = s;
    this.sem.acquire();
  }
  @Override
  public void finalize() {
    this.sem.release();
  }
}
class Foo {
  private Semaphore sem = new Semaphore(5);
  void print(int x) throws Exception {
    new Permit(this.sem);
    if (x > 1000) {
      throw new Exception("Too large!");
    }
    System.out.printf("x = %d", x);
  }
}

See how beautiful the code is inside method Foo.print(). We just create an instance of class Permit and it immediately acquires a new permit at the semaphore. Then we exit the method print(), either by exception or in the normal way, and the method Permit.finalize() releases the permit.

Elegant, isn’t it? Yes, it is, but it won’t work in Java.

It won’t work because, unlike C++, Java doesn’t destroy objects when their scope of visibility is closed. The object of class Permit won’t be destroyed when we exit the method print(). It will be destroyed eventually but we don’t know when exactly. Most likely it will be destroyed way after all permits in the semaphore got acquired and we get blocked.

There is a solution in Java too. It is not as elegant as the one from C++, but it does work. Here it is:

class Permit implements Closeable {
  private Semaphore sem;
  Permit(Semaphore s) {
    this.sem = s;
  }
  @Override
  public void close() {
    this.sem.release();
  }
  public Permit acquire() {
    this.sem.acquire();
    return this;
  }
}
class Foo {
  private Semaphore sem = new Semaphore(5);
  void print(int x) throws Exception {
    try (Permit p = new Permit(this.sem).acquire()) {
      if (x > 1000) {
        throw new Exception("Too large!");
      }
      System.out.printf("x = %d", x);
    }
  }
}

Pay attention to the try block and to the Closeable interface that the class Permit now implements. The object p will be “closed” when the try block exits. It may exit either at the end, or by the return or throw statements. In either case Permit.close() will be called: this is how try-with-resources works in Java.

I introduced method acquire() and moved sem.acquire() out of the Permit constructor because I believe that constructors must be code-free.

To summarize, RAII is a perfect design pattern approach when you deal with resources that may leak. Even though Java doesn’t have it out of the box we can implement it via try-with-resources and Closeable.

Resource Acquisition Is Initialization (RAII) is a design idea introduced in C++ by Bjarne Stroustrup for exception-safe resource management. Thanks to garbage collection Java doesn’t have this feature, but we can implement something similar, using try-with-resources.

At Sachem Farm (1998) by John Huddles
At Sachem Farm (1998) by John Huddles

The problem RAII is solving is obvious; have a look at this code (I’m sure you know what Semaphore is and how it works in Java):

class Foo {
  private Semaphore sem = new Semaphore(5);
  void print(int x) throws Exception {
    this.sem.acquire();
    if (x > 1000) {
      throw new Exception("Too large!");
    }
    System.out.printf("x = %d", x);
    this.sem.release();
  }
}

The code is rather primitive and doesn’t do anything useful, but you most probably get the idea: the method print(), if being called from multiple parallel threads, will allow only five of them to print in parallel. Sometimes it will not allow some of them to print and will throw an exception if x is bigger than 1000.

The problem with this code is—resource leakage. Each print() call with x larger than 1000 will take one permit from the semaphore and won’t return it. In five calls with exceptions the semaphore will be empty and all other threads won’t print anything.

What is the solution? Here it is:

class Foo {
  private Semaphore sem = new Semaphore(5);
  void print(int x) throws Exception {
    this.sem.acquire();
    if (x > 1000) {
      this.sem.release();
      throw new Exception("Too large!");
    }
    System.out.printf("x = %d", x);
    this.sem.release();
  }
}

We must release the permit before we throw the exception.

However, there is another problem that shows up: code duplication. We release the permit in two places. If we add more throw instructions we will also have to add more sem.release() calls.

A very elegant solution was introduced in C++ and is called RAII. This is how it would look in Java:

class Permit {
  private Semaphore sem;
  Permit(Semaphore s) {
    this.sem = s;
    this.sem.acquire();
  }
  @Override
  public void finalize() {
    this.sem.release();
  }
}
class Foo {
  private Semaphore sem = new Semaphore(5);
  void print(int x) throws Exception {
    new Permit(this.sem);
    if (x > 1000) {
      throw new Exception("Too large!");
    }
    System.out.printf("x = %d", x);
  }
}

See how beautiful the code is inside method Foo.print(). We just create an instance of class Permit and it immediately acquires a new permit at the semaphore. Then we exit the method print(), either by exception or in the normal way, and the method Permit.finalize() releases the permit.

Elegant, isn’t it? Yes, it is, but it won’t work in Java.

It won’t work because, unlike C++, Java doesn’t destroy objects when their scope of visibility is closed. The object of class Permit won’t be destroyed when we exit the method print(). It will be destroyed eventually but we don’t know when exactly. Most likely it will be destroyed way after all permits in the semaphore got acquired and we get blocked.

There is a solution in Java too. It is not as elegant as the one from C++, but it does work. Here it is:

class Permit implements Closeable {
  private Semaphore sem;
  Permit(Semaphore s) {
    this.sem = s;
  }
  @Override
  public void close() {
    this.sem.release();
  }
  public Permit acquire() {
    this.sem.acquire();
    return this;
  }
}
class Foo {
  private Semaphore sem = new Semaphore(5);
  void print(int x) throws Exception {
    try (Permit p = new Permit(this.sem).acquire()) {
      if (x > 1000) {
        throw new Exception("Too large!");
      }
      System.out.printf("x = %d", x);
    }
  }
}

Pay attention to the try block and to the Closeable interface that the class Permit now implements. The object p will be “closed” when the try block exits. It may exit either at the end, or by the return or throw statements. In either case Permit.close() will be called: this is how try-with-resources works in Java.

I introduced method acquire() and moved sem.acquire() out of the Permit constructor because I believe that constructors must be code-free.

To summarize, RAII is a perfect design pattern approach when you deal with resources that may leak. Even though Java doesn’t have it out of the box we can implement it via try-with-resources and Closeable.

© Yegor Bugayenko 2014–2018

How I Would Re-design equals()

QR code

How I Would Re-design equals()

  • Copenhagen, Denmark
  • comments

I want to rant a bit about Java design, in particular about the methods Object.equals() and Comparable.compareTo(). I’ve hated them for years, because, no matter how hard I try to like them, the code inside looks ugly. Now I know what exactly is wrong and how I would design this “object-to-object comparing” mechanism better.

L'ultimo capodanno (1998) by Marco Risi
L'ultimo capodanno (1998) by Marco Risi

Say we have a simple primitive class Weight, objects of which represent the weight of something in kilos:

class Weight {
  private int kilos;
  Weight(int k) {
    this.kilos = k;
  }
}

Next, we want two objects of the same weight to be equal to each other:

new Weight(15).equals(new Weight(15));

Here is how such a method may look:

class Weight {
  private int kilos;
  Weight(int k) {
    this.kilos = k;
  }
  public boolean equals(Object obj) {
    if (!(obj instanceof Weight)) {
      return false;
    }
    Weight weight = Weight.class.cast(obj);
    return weight.kilos == this.kilos;
  }
}

The ugly part here is, first of all, the type casting with instanceof. The second problem is that we touch the internals of the incoming object. This design makes polymorphic behavior of the Weight impossible. We simply can’t pass anything else to the equals() method, besides an instance of the class Weight. We can’t turn it into an interface and introduce multiple implementations of it:

interface Weight {
  boolean equals(Object obj);
}

This code will not work:

class DefaultWeight implements Weight {
  // attribute and ctor skipped
  public boolean equals(Object obj) {
    if (!(obj instanceof Weight)) {
      return false;
    }
    Weight weight = Weight.class.cast(obj);
    return weight.kilos == this.kilos; // error here!
  }
}

The problem is that one object decides for the other whether they are equal. This inevitably leads to a necessity to touch private attributes in order to do the actual comparison.

What is the solution?

This is what I’m offering. Any comparison, no matter what types we are talking about, is about comparing two digital values. Either we compare a weight with a weight, text with text, or a user with a user—our CPUs can only compare numbers. Thus, we introduce a new interface Digitizable:

interface Digitizable {
  byte[] digits();
}

Next, we introduce a new class Comparison, which is the comparison of two streams of bytes (I’m not sure the code is perfect, I tested it here, feel free to improve and contribute with a pull request):

class Comparison<T extends Digitizable> {
  private T lt;
  private T rt;
  Comparison(T left, T right) {
    this.lt = left;
    this.rt = right;
  }
  int value() {
    final byte[] left = this.lt.digits();
    final byte[] right = this.rt.digits();
    int result = 0;
    int max = Math.max(left.length, right.length);
    for (int idx = max; idx > 0; --idx) {
      byte lft = 0;
      if (idx <= left.length) {
        lft = left[max - idx];
      }
      byte rht = 0;
      if (idx <= right.length) {
        rht = right[max - idx];
      }
      result = lft - rht;
      if (result != 0) {
        break;
      }
    }
    return (int) Math.signum(result);
  }
}

Now, we need Weight to implement Digitizable:

class Weight implements Digitizable {
  private int kilos;
  Weight(int k) {
    this.kilos = k;
  }
  @Override
  public byte[] digits() {
    return ByteBuffer.allocate(4)
      .putInt(this.kilos).array();
  }
}

Finally, this is how we compare them:

int v = new Comparison<Weight>(
  new Weight(400), new Weight(500)
).value();

This v will either be -1, 0, or 1. In this particular case it will be -1, because 400 is less than 500.

No more violation of encapsulation, no more type casting, no more ugly code inside those equals() and compareTo() methods. The class Comparison will work with all possible types. All our objects need to do in order to become comparable is to implement Digitizable and “provide” their bytes for inspection/comparison.

This approach is actually very close to the printers I described earlier.

I want to rant a bit about Java design, in particular about the methods Object.equals() and Comparable.compareTo(). I’ve hated them for years, because, no matter how hard I try to like them, the code inside looks ugly. Now I know what exactly is wrong and how I would design this “object-to-object comparing” mechanism better.

L'ultimo capodanno (1998) by Marco Risi
L'ultimo capodanno (1998) by Marco Risi

Say we have a simple primitive class Weight, objects of which represent the weight of something in kilos:

class Weight {
  private int kilos;
  Weight(int k) {
    this.kilos = k;
  }
}

Next, we want two objects of the same weight to be equal to each other:

new Weight(15).equals(new Weight(15));

Here is how such a method may look:

class Weight {
  private int kilos;
  Weight(int k) {
    this.kilos = k;
  }
  public boolean equals(Object obj) {
    if (!(obj instanceof Weight)) {
      return false;
    }
    Weight weight = Weight.class.cast(obj);
    return weight.kilos == this.kilos;
  }
}

The ugly part here is, first of all, the type casting with instanceof. The second problem is that we touch the internals of the incoming object. This design makes polymorphic behavior of the Weight impossible. We simply can’t pass anything else to the equals() method, besides an instance of the class Weight. We can’t turn it into an interface and introduce multiple implementations of it:

interface Weight {
  boolean equals(Object obj);
}

This code will not work:

class DefaultWeight implements Weight {
  // attribute and ctor skipped
  public boolean equals(Object obj) {
    if (!(obj instanceof Weight)) {
      return false;
    }
    Weight weight = Weight.class.cast(obj);
    return weight.kilos == this.kilos; // error here!
  }
}

The problem is that one object decides for the other whether they are equal. This inevitably leads to a necessity to touch private attributes in order to do the actual comparison.

What is the solution?

This is what I’m offering. Any comparison, no matter what types we are talking about, is about comparing two digital values. Either we compare a weight with a weight, text with text, or a user with a user—our CPUs can only compare numbers. Thus, we introduce a new interface Digitizable:

interface Digitizable {
  byte[] digits();
}

Next, we introduce a new class Comparison, which is the comparison of two streams of bytes (I’m not sure the code is perfect, I tested it here, feel free to improve and contribute with a pull request):

class Comparison<T extends Digitizable> {
  private T lt;
  private T rt;
  Comparison(T left, T right) {
    this.lt = left;
    this.rt = right;
  }
  int value() {
    final byte[] left = this.lt.digits();
    final byte[] right = this.rt.digits();
    int result = 0;
    int max = Math.max(left.length, right.length);
    for (int idx = max; idx > 0; --idx) {
      byte lft = 0;
      if (idx <= left.length) {
        lft = left[max - idx];
      }
      byte rht = 0;
      if (idx <= right.length) {
        rht = right[max - idx];
      }
      result = lft - rht;
      if (result != 0) {
        break;
      }
    }
    return (int) Math.signum(result);
  }
}

Now, we need Weight to implement Digitizable:

class Weight implements Digitizable {
  private int kilos;
  Weight(int k) {
    this.kilos = k;
  }
  @Override
  public byte[] digits() {
    return ByteBuffer.allocate(4)
      .putInt(this.kilos).array();
  }
}

Finally, this is how we compare them:

int v = new Comparison<Weight>(
  new Weight(400), new Weight(500)
).value();

This v will either be -1, 0, or 1. In this particular case it will be -1, because 400 is less than 500.

No more violation of encapsulation, no more type casting, no more ugly code inside those equals() and compareTo() methods. The class Comparison will work with all possible types. All our objects need to do in order to become comparable is to implement Digitizable and “provide” their bytes for inspection/comparison.

This approach is actually very close to the printers I described earlier.

© Yegor Bugayenko 2014–2018

Object-Oriented Declarative Input/Output in Cactoos

QR code

Object-Oriented Declarative Input/Output in Cactoos

  • Dnipro, Ukraine
  • comments

badge

Cactoos is a library of object-oriented Java primitives we started to work on just a few weeks ago. The intent was to propose a clean and more declarative alternative to JDK, Guava, Apache Commons, and others. Instead of calling static procedures we want to use objects, the way they are supposed to be used. Let’s see how input/output works in a pure object-oriented fashion.

Disclaimer: The version I’m using at the time of writing is 0.9. Later versions may have different names of classes and a totally different design.

Let’s say you want to read a file. This is how you would do it with the static method readAllBytes() from the utility class Files in JDK7:

byte[] content = Files.readAllBytes(
  new File("/tmp/photo.jpg").toPath()
);

This code is very imperative—it reads the file content right here and now, placing it into the array.

This is how you do it with Cactoos:

Bytes source = new InputAsBytes(
  new FileAsInput(
    new File("/tmp/photo.jpg")
  )
);

Pay attention—there are no method calls yet. Just three constructors of three classes that compose a bigger object. The object source is of type Bytes and represents the content of the file. To get that content out of it we call its method asBytes():

bytes[] content = source.asBytes();

This is the moment when the file system is touched. This approach, as you can see, is absolutely declarative and thanks to that possesses all the benefits of object orientation.

Here is another example. Say you want to write some text into a file. Here is how you do it in Cactoos. First you need the Input:

Input input = new BytesAsInput(
  new TextAsBytes(
    new StringAsText(
      "Hello, world!"
    )
  )
);

Then you need the Output:

Output output = new FileAsOutput(
  new File("/tmp/hello.txt")
);

Now, we want to copy the input to the output. There is no “copy” operation in pure OOP. Moreover, there must be no operations at all. Just objects. We have a class named TeeInput, which is an Input that copies everything you read from it to the Output, similar to what TeeInputStream from Apache Commons does, but encapsulated. So we don’t copy, we create an Input that will copy if you touch it:

Input tee = new TeeInput(input, output);

Now, we have to “touch” it. And we have to touch every single byte of it, in order to make sure they all are copied. If we just read() the first byte, only one byte will be copied to the file. The best way to touch them all is to calculate the size of the tee object, going byte by byte. We have an object for it, called LengthOfInput. It encapsulates an Input and behaves like its length in bytes:

Scalar<Long> length = new LengthOfInput(tee);

Then we take the value out of it and the file writing operation takes place:

long len = length.value();

Thus, the entire operation of writing the string to the file will look like this:

new LengthOfInput(
  new TeeInput(
    new BytesAsInput(
      new TextAsBytes(
        new StringAsText(
          "Hello, world!"
        )
      )
    ),
    new FileAsOutput(
      new File("/tmp/hello.txt")
    )
  )
).value(); // happens here

This is its procedural alternative from JDK7:

Files.write(
  new File("/tmp/hello.txt").toPath(),
  "Hello, world!".getBytes()
);

“Why is object-oriented better, even though it’s longer?” I hear you ask. Because it perfectly decouples concepts, while the procedural one keeps them together.

Let’s say, you are designing a class that is supposed to encrypt some text and save it to a file. Here is how you would design it the procedural way (not a real encryption, of course):

class Encoder {
  private final File target;
  Encoder(final File file) {
    this.target = file;
  }
  void encode(String text) {
    Files.write(
      this.target,
      text.replaceAll("[a-z]", "*")
    );
  }
}

Works fine, but what will happen when you decide to extend it to also write to an OutputStream? How will you modify this class? How ugly will it look after that? That’s because the design is not object-oriented.

This is how you would do the same design, in an object-oriented way, with Cactoos:

class Encoder {
  private final Output target;
  Encoder(final File file) {
    this(new FileAsOutput(file));
  }
  Encoder(final Output output) {
    this.target = output;
  }
  void encode(String text) {
    new LengthOfInput(
      new TeeInput(
        new BytesAsInput(
          new TextAsBytes(
            new StringAsText(
              text.replaceAll("[a-z]", "*")
            )
          )
        ),
        this.target
      )
    ).value();
  }
}

What do we do with this design if we want OutputStream to be accepted? We just add one secondary constructor:

class Encoder {
  Encoder(final OutputStream stream) {
    this(new OutputStreamAsOutput(stream));
  }
}

Done. That’s how easy and elegant it is.

That’s because concepts are perfectly separated and functionality is encapsulated. In the procedural example the behavior of the object is located outside of it, in the method encode(). The file itself doesn’t know how to write, some outside procedure Files.write() knows that instead.

To the contrary, in the object-oriented design the FileAsOutput knows how to write, and nobody else does. The file writing functionality is encapsulated and this makes it possible to decorate the objects in any possible way, creating reusable and replaceable composite objects.

Do you see the beauty of OOP now?

badge

Cactoos is a library of object-oriented Java primitives we started to work on just a few weeks ago. The intent was to propose a clean and more declarative alternative to JDK, Guava, Apache Commons, and others. Instead of calling static procedures we want to use objects, the way they are supposed to be used. Let’s see how input/output works in a pure object-oriented fashion.

Disclaimer: The version I’m using at the time of writing is 0.9. Later versions may have different names of classes and a totally different design.

Let’s say you want to read a file. This is how you would do it with the static method readAllBytes() from the utility class Files in JDK7:

byte[] content = Files.readAllBytes(
  new File("/tmp/photo.jpg").toPath()
);

This code is very imperative—it reads the file content right here and now, placing it into the array.

This is how you do it with Cactoos:

Bytes source = new InputAsBytes(
  new FileAsInput(
    new File("/tmp/photo.jpg")
  )
);

Pay attention—there are no method calls yet. Just three constructors of three classes that compose a bigger object. The object source is of type Bytes and represents the content of the file. To get that content out of it we call its method asBytes():

bytes[] content = source.asBytes();

This is the moment when the file system is touched. This approach, as you can see, is absolutely declarative and thanks to that possesses all the benefits of object orientation.

Here is another example. Say you want to write some text into a file. Here is how you do it in Cactoos. First you need the Input:

Input input = new BytesAsInput(
  new TextAsBytes(
    new StringAsText(
      "Hello, world!"
    )
  )
);

Then you need the Output:

Output output = new FileAsOutput(
  new File("/tmp/hello.txt")
);

Now, we want to copy the input to the output. There is no “copy” operation in pure OOP. Moreover, there must be no operations at all. Just objects. We have a class named TeeInput, which is an Input that copies everything you read from it to the Output, similar to what TeeInputStream from Apache Commons does, but encapsulated. So we don’t copy, we create an Input that will copy if you touch it:

Input tee = new TeeInput(input, output);

Now, we have to “touch” it. And we have to touch every single byte of it, in order to make sure they all are copied. If we just read() the first byte, only one byte will be copied to the file. The best way to touch them all is to calculate the size of the tee object, going byte by byte. We have an object for it, called LengthOfInput. It encapsulates an Input and behaves like its length in bytes:

Scalar<Long> length = new LengthOfInput(tee);

Then we take the value out of it and the file writing operation takes place:

long len = length.value();

Thus, the entire operation of writing the string to the file will look like this:

new LengthOfInput(
  new TeeInput(
    new BytesAsInput(
      new TextAsBytes(
        new StringAsText(
          "Hello, world!"
        )
      )
    ),
    new FileAsOutput(
      new File("/tmp/hello.txt")
    )
  )
).value(); // happens here

This is its procedural alternative from JDK7:

Files.write(
  new File("/tmp/hello.txt").toPath(),
  "Hello, world!".getBytes()
);

“Why is object-oriented better, even though it’s longer?” I hear you ask. Because it perfectly decouples concepts, while the procedural one keeps them together.

Let’s say, you are designing a class that is supposed to encrypt some text and save it to a file. Here is how you would design it the procedural way (not a real encryption, of course):

class Encoder {
  private final File target;
  Encoder(final File file) {
    this.target = file;
  }
  void encode(String text) {
    Files.write(
      this.target,
      text.replaceAll("[a-z]", "*")
    );
  }
}

Works fine, but what will happen when you decide to extend it to also write to an OutputStream? How will you modify this class? How ugly will it look after that? That’s because the design is not object-oriented.

This is how you would do the same design, in an object-oriented way, with Cactoos:

class Encoder {
  private final Output target;
  Encoder(final File file) {
    this(new FileAsOutput(file));
  }
  Encoder(final Output output) {
    this.target = output;
  }
  void encode(String text) {
    new LengthOfInput(
      new TeeInput(
        new BytesAsInput(
          new TextAsBytes(
            new StringAsText(
              text.replaceAll("[a-z]", "*")
            )
          )
        ),
        this.target
      )
    ).value();
  }
}

What do we do with this design if we want OutputStream to be accepted? We just add one secondary constructor:

class Encoder {
  Encoder(final OutputStream stream) {
    this(new OutputStreamAsOutput(stream));
  }
}

Done. That’s how easy and elegant it is.

That’s because concepts are perfectly separated and functionality is encapsulated. In the procedural example the behavior of the object is located outside of it, in the method encode(). The file itself doesn’t know how to write, some outside procedure Files.write() knows that instead.

To the contrary, in the object-oriented design the FileAsOutput knows how to write, and nobody else does. The file writing functionality is encapsulated and this makes it possible to decorate the objects in any possible way, creating reusable and replaceable composite objects.

Do you see the beauty of OOP now?

© Yegor Bugayenko 2014–2018

Single Statement Unit Tests

QR code

Single Statement Unit Tests

  • Odessa, Ukraine
  • comments

Many articles and books have already been written about unit testing patterns and anti-patterns. I want to add one more recommendation which, I believe, can help us make our tests, and our production code, more object-oriented. Here it is: a test method must contain nothing but a single assert.

Bullet (1996) by Julien Temple
Bullet (1996) by Julien Temple

Look at this test method from RandomStreamTest from OpenJDK 8, created by Brian Goetz:

@Test
public void testIntStream() {
  final long seed = System.currentTimeMillis();
  final Random r1 = new Random(seed);
  final int[] a = new int[SIZE];
  for (int i=0; i < SIZE; i++) {
    a[i] = r1.nextInt();
  }
  final Random r2 = new Random(seed);
  final int[] b = r2.ints().limit(SIZE).toArray();
  assertEquals(a, b);
}

There are two parts in this method: the algorithm and the assertion. The algorithm prepares two arrays of integers and the assertion compares them and throws AssertionError if they are not equal.

I’m saying that the first part, the algorithm, is the one we should try to avoid. The only thing we must have is the assertion. Here is how I would re-design this test method:

@Test
public void testIntStream() {
  final long seed = System.currentTimeMillis();
  assertEquals(
    new ArrayFromRandom(
      new Random(seed)
    ).toArray(SIZE),
    new Random(seed).ints().limit(SIZE).toArray()
  );
}
private static class ArrayFromRandom {
  private final Random random;
  ArrayFromRandom(Random r) {
    this.random = r;
  }
  int[] toArray(int s) {
    final int[] a = new int[s];
    for (int i=0; i < s; i++) {
      a[i] = this.random.nextInt();
    }
    return a;
  }
}

If Java had monikers this code would look even more elegant:

@Test
public void testIntStream() {
  assertEquals(
    new ArrayFromRandom(
      new Random(System.currentTimeMillis() as seed)
    ).toArray(SIZE),
    new Random(seed).ints().limit(SIZE).toArray()
  );
}

As you can see, there is only one “statement” in this method: assertEquals().

Hamcrest with its assertThat() and its collection of basic matchers is a perfect instrument to make our single-statement test methods even more cohesive and readable.

There are a number of practical benefits of this principle, if we agree to follow it:

  • Reusability. The classes we will have to create for test assertions will be reusable in other test methods and test cases. Just as, in the example above, ArrayFromRandom could be used somewhere else. Similarly, Hamcrest matchers may and will constitute a library of reusable test components.

  • Brevity. Since it will be rather difficult to create a long test method when it only has a single assert, you and your fellow programmers will inevitably write shorter and more readable code.

  • Readability. With a single assert it will always be obvious what the intent of the test method is. It will start with the intent declaration while all other lower level details will be indented.

  • Immutability. It will be almost impossible to have setters in the production code if test methods have no place for algorithmic code. You inevitably will create immutable objects to make them testable with a single assert.

The biggest benefit we get when this principle is applied to our tests is that they become declarative and object-oriented, instead of being algorithmic, imperative, and procedural.

Many articles and books have already been written about unit testing patterns and anti-patterns. I want to add one more recommendation which, I believe, can help us make our tests, and our production code, more object-oriented. Here it is: a test method must contain nothing but a single assert.

Bullet (1996) by Julien Temple
Bullet (1996) by Julien Temple

Look at this test method from RandomStreamTest from OpenJDK 8, created by Brian Goetz:

@Test
public void testIntStream() {
  final long seed = System.currentTimeMillis();
  final Random r1 = new Random(seed);
  final int[] a = new int[SIZE];
  for (int i=0; i < SIZE; i++) {
    a[i] = r1.nextInt();
  }
  final Random r2 = new Random(seed);
  final int[] b = r2.ints().limit(SIZE).toArray();
  assertEquals(a, b);
}

There are two parts in this method: the algorithm and the assertion. The algorithm prepares two arrays of integers and the assertion compares them and throws AssertionError if they are not equal.

I’m saying that the first part, the algorithm, is the one we should try to avoid. The only thing we must have is the assertion. Here is how I would re-design this test method:

@Test
public void testIntStream() {
  final long seed = System.currentTimeMillis();
  assertEquals(
    new ArrayFromRandom(
      new Random(seed)
    ).toArray(SIZE),
    new Random(seed).ints().limit(SIZE).toArray()
  );
}
private static class ArrayFromRandom {
  private final Random random;
  ArrayFromRandom(Random r) {
    this.random = r;
  }
  int[] toArray(int s) {
    final int[] a = new int[s];
    for (int i=0; i < s; i++) {
      a[i] = this.random.nextInt();
    }
    return a;
  }
}

If Java had monikers this code would look even more elegant:

@Test
public void testIntStream() {
  assertEquals(
    new ArrayFromRandom(
      new Random(System.currentTimeMillis() as seed)
    ).toArray(SIZE),
    new Random(seed).ints().limit(SIZE).toArray()
  );
}

As you can see, there is only one “statement” in this method: assertEquals().

Hamcrest with its assertThat() and its collection of basic matchers is a perfect instrument to make our single-statement test methods even more cohesive and readable.

There are a number of practical benefits of this principle, if we agree to follow it:

  • Reusability. The classes we will have to create for test assertions will be reusable in other test methods and test cases. Just as, in the example above, ArrayFromRandom could be used somewhere else. Similarly, Hamcrest matchers may and will constitute a library of reusable test components.

  • Brevity. Since it will be rather difficult to create a long test method when it only has a single assert, you and your fellow programmers will inevitably write shorter and more readable code.

  • Readability. With a single assert it will always be obvious what the intent of the test method is. It will start with the intent declaration while all other lower level details will be indented.

  • Immutability. It will be almost impossible to have setters in the production code if test methods have no place for algorithmic code. You inevitably will create immutable objects to make them testable with a single assert.

The biggest benefit we get when this principle is applied to our tests is that they become declarative and object-oriented, instead of being algorithmic, imperative, and procedural.

© Yegor Bugayenko 2014–2018

Traits and Mixins Are Not OOP

QR code

Traits and Mixins Are Not OOP

  • Odessa, Ukraine
  • comments

Let me say right off the bat that the features we will discuss here are pure poison brought to object-oriented programming by those who desperately needed a lobotomy, just like David West suggested in his Object Thinking book. These features have different names, but the most common ones are traits and mixins. I seriously can’t understand how we can still call programming object-oriented when it has these features.

Fear and Loathing in Las Vegas (1998) by Terry Gilliam
Fear and Loathing in Las Vegas (1998) by Terry Gilliam

First, here’s how they work in a nutshell. Let’s use Ruby modules as a sample implementation. Say that we have a class Book:

class Book
  def initialize(title)
    @title = title
  end
end

Now, we want class Book to use a static method (a procedure) that does something useful. We may either define it in a utility class and let Book call it:

class TextUtils
  def self.caps(text)
    text.split.map(&:capitalize).join(' ')
  end
end
class Book
  def print
    puts "My title is #{TextUtils.caps(@title)}"
  end
end

Or we may make it even more “convenient” and extend our module in order to access its methods directly:

module TextModule
  def caps(text)
    text.split.map(&:capitalize).join(' ')
  end
end
class Book
  extend TextModule
  def print
    puts "My title is #{caps(@title)}"
  end
end

It seems nice—if you don’t understand the difference between object-oriented programming and static methods. Moreover, if we forget OOP purity for a minute, this approach actually looks less readable to me, even though it has fewer characters; it’s difficult to understand where the method caps() is coming from when it’s called just like #{caps(@title)} instead of #{TextUtils.caps(@title)}. Don’t you think?

Mixins start to play their role better when we include them. We can combine them to construct the behavior of the class we’re looking for. Let’s create two mixins. The first one will be called PlainMixin and will print the title of the book the way it is, and the second one will be called CapsMixin and will capitalize what’s already printed:

module CapsMixin
  def to_s
    super.to_s.split.map(&:capitalize).join(' ')
  end
end
module PlainMixin
  def to_s
    @title
  end
end
class Book
  def initialize(title)
    @title = title
  end
  include CapsMixin, PlainMixin
  def print
    puts "My title is #{self}"
  end
end

Calling Book without the included mixin will print its title the way it is. Once we add the include statement, the behavior of to_s is overridden and method print produces a different result. We can combine mixins to produce the required functionality. For example, we can add one more, which will abbreviate the title to 16 characters:

module AbbrMixin
  def to_s
    super.to_s.gsub(/^(.{16,}?).*$/m,'\1...')
  end
end
class Book
  def initialize(title)
    @title = title
  end
  include AbbrMixin, CapsMixin, PlainMixin
  def print
    puts "My title is #{self}"
  end
end

I’m sure you already understand that they both have access to the private attribute @title of class Book. They actually have full access to everything in the class. They literally are “pieces of code” that we inject into the class to make it more powerful and complex. What’s wrong with this approach?

It’s the same issue as with annotations, DTOs, getters, and utility classes—they tear objects apart and place pieces of functionality in places where objects don’t see them.

In the case of mixins, the functionality is in the Ruby modules, which make assumptions about the internal structure of Book and further assume that the programmer will still understand what’s in Book after the internal structure changes. Such assumptions completely violate the very idea of encapsulation.

Such a tight coupling between mixins and object private structure leads to nothing but unmaintainable and difficult to understand code.

The very obvious alternatives to mixins are composable decorators. Take a look at the example given in the article:

Text text = new AllCapsText(
  new TrimmedText(
    new PrintableText(
      new TextInFile(new File("/tmp/a.txt"))
    )
  )
);

Doesn’t it look very similar to what we were doing above with Ruby mixins?

However, unlike mixins, decorators leave objects small and cohesive, layering extra functionality on top of them. Mixins do the opposite—they make objects more complex and, thanks to that, less readable and maintainable.

I honestly believe they are just poison. Whoever invented them was a long ways from understanding the philosophy of object-oriented design.

Let me say right off the bat that the features we will discuss here are pure poison brought to object-oriented programming by those who desperately needed a lobotomy, just like David West suggested in his Object Thinking book. These features have different names, but the most common ones are traits and mixins. I seriously can’t understand how we can still call programming object-oriented when it has these features.

Fear and Loathing in Las Vegas (1998) by Terry Gilliam
Fear and Loathing in Las Vegas (1998) by Terry Gilliam

First, here’s how they work in a nutshell. Let’s use Ruby modules as a sample implementation. Say that we have a class Book:

class Book
  def initialize(title)
    @title = title
  end
end

Now, we want class Book to use a static method (a procedure) that does something useful. We may either define it in a utility class and let Book call it:

class TextUtils
  def self.caps(text)
    text.split.map(&:capitalize).join(' ')
  end
end
class Book
  def print
    puts "My title is #{TextUtils.caps(@title)}"
  end
end

Or we may make it even more “convenient” and extend our module in order to access its methods directly:

module TextModule
  def caps(text)
    text.split.map(&:capitalize).join(' ')
  end
end
class Book
  extend TextModule
  def print
    puts "My title is #{caps(@title)}"
  end
end

It seems nice—if you don’t understand the difference between object-oriented programming and static methods. Moreover, if we forget OOP purity for a minute, this approach actually looks less readable to me, even though it has fewer characters; it’s difficult to understand where the method caps() is coming from when it’s called just like #{caps(@title)} instead of #{TextUtils.caps(@title)}. Don’t you think?

Mixins start to play their role better when we include them. We can combine them to construct the behavior of the class we’re looking for. Let’s create two mixins. The first one will be called PlainMixin and will print the title of the book the way it is, and the second one will be called CapsMixin and will capitalize what’s already printed:

module CapsMixin
  def to_s
    super.to_s.split.map(&:capitalize).join(' ')
  end
end
module PlainMixin
  def to_s
    @title
  end
end
class Book
  def initialize(title)
    @title = title
  end
  include CapsMixin, PlainMixin
  def print
    puts "My title is #{self}"
  end
end

Calling Book without the included mixin will print its title the way it is. Once we add the include statement, the behavior of to_s is overridden and method print produces a different result. We can combine mixins to produce the required functionality. For example, we can add one more, which will abbreviate the title to 16 characters:

module AbbrMixin
  def to_s
    super.to_s.gsub(/^(.{16,}?).*$/m,'\1...')
  end
end
class Book
  def initialize(title)
    @title = title
  end
  include AbbrMixin, CapsMixin, PlainMixin
  def print
    puts "My title is #{self}"
  end
end

I’m sure you already understand that they both have access to the private attribute @title of class Book. They actually have full access to everything in the class. They literally are “pieces of code” that we inject into the class to make it more powerful and complex. What’s wrong with this approach?

It’s the same issue as with annotations, DTOs, getters, and utility classes—they tear objects apart and place pieces of functionality in places where objects don’t see them.

In the case of mixins, the functionality is in the Ruby modules, which make assumptions about the internal structure of Book and further assume that the programmer will still understand what’s in Book after the internal structure changes. Such assumptions completely violate the very idea of encapsulation.

Such a tight coupling between mixins and object private structure leads to nothing but unmaintainable and difficult to understand code.

The very obvious alternatives to mixins are composable decorators. Take a look at the example given in the article:

Text text = new AllCapsText(
  new TrimmedText(
    new PrintableText(
      new TextInFile(new File("/tmp/a.txt"))
    )
  )
);

Doesn’t it look very similar to what we were doing above with Ruby mixins?

However, unlike mixins, decorators leave objects small and cohesive, layering extra functionality on top of them. Mixins do the opposite—they make objects more complex and, thanks to that, less readable and maintainable.

I honestly believe they are just poison. Whoever invented them was a long ways from understanding the philosophy of object-oriented design.

© Yegor Bugayenko 2014–2018

How to Handle the Problem of Too Many Classes

QR code

How to Handle the Problem of Too Many Classes

  • Odessa, Ukraine
  • comments

During nearly every presentation in which I explain my view of object-oriented programming, there is someone who shares a comment like this: “If we follow your advice, we will have so many small classes.” And my answer is always the same: “Of course we will, and that’s great!” I honestly believe that even if you can’t consider having “a lot of classes” a virtue, you can’t call it a drawback of any truly object-oriented code either. However, there may come a point when classes become a problem; let’s see when, how, and what to do about that.

El día de la bestia (1995) by Álex de la Iglesia
El día de la bestia (1995) by Álex de la Iglesia

There were a number of “rules” previously mentioned that, if applied, would obviously lead to a large number of classes, including: a) all public methods must be declared in interfaces; b) objects must not have more than four attributes (Section 2.1 of Elegant Objects); c) static methods are not allowed; d) constructors must be code-free; e) objects must expose fewer than five public methods (Section 3.1 of Elegant Objects).

The biggest concern, of course, is maintainability: “If, instead of 50 longer classes, we had 300 shorter ones, then the code would be way less readable.” This will most certainly happen if you design them wrong.

Types (or classes) in OOP constitute your vocabulary, which explains the world around your code—the world your code lives in. The richer the vocabulary, the more powerful your code. The more types you have, the better you can understand and explain the world.

If your vocabulary is big enough, you will say something like:

Read the book that is on the table.

With a much smaller vocabulary, the same phrase would sound like:

Do it with the thing that is on that thing.

Obviously, it’s easier to read and understand the first phrase. The same occurs with types in OOP: the more of them you have at your disposal, the more expressive, bright, and readable your code is.

Unfortunately, Java and many other languages are not designed with this concept in mind. Packages, modules, and namespaces don’t really help, and we usually end up with names like AbstractCookieValueMethodArgumentResolver (Spring) or CombineFileRecordReaderWrapper (Hadoop). We’re trying to pack as many semantics into class names as possible so their users won’t doubt for a second. Then we’re trying to put as many methods into one class as possible to make life easier for users; they will use their IDE hints to find the right one.

This is anything but OOP.

If your code is object-oriented, your classes must be small, their names must be nouns, and their method names must be just one word. Here is what I do in my code to make that happen:

Interfaces are nouns. For example, Request, Directive, or Domain. There are no exceptions. Types (also known as interfaces in Java) are the core part of my vocabulary; they have to be nouns.

Classes are prefixed. My classes always implement interfaces. Thanks to that, I can say they always are requests, directives, or domains. And I always want their users to remember that. Prefixes help. For example, RqBuffered is a buffered request, RqSimple is a simple request, RqLive is a request that represents a “live” HTTP connection, and RqWithHeader is a request with an extra header.

An alternative approach is to use the type name as the central part of the class name and add a prefix that explains implementation details. For example, DyDomain is a domain that persists its data in DynamoDB. Once you know what that Dy prefix is for, you can easily understand what DyUser and DyBase are about.

In a medium-sized application or a library, there will be as many as 10 to 15 prefixes you will have to remember, no more. For example, in the Takes Framework, there are 24,000 lines of code, 410 Java files, and 10 prefixes: Bc, Cc, Tk, Rq, Rs, Fb, Fk, Hm, Ps, and Xe. Not so difficult to remember what they mean, right?

Among all 240 classes, the longest name is RqWithDefaultHeader.

I find this approach to class naming rather convenient. I used it in these open source projects (in GitHub): yegor256/takes (10 prefixes), yegor256/jare (5 prefixes), yegor256/rultor (6 prefixes), and yegor256/wring (5 prefixes).

During nearly every presentation in which I explain my view of object-oriented programming, there is someone who shares a comment like this: “If we follow your advice, we will have so many small classes.” And my answer is always the same: “Of course we will, and that’s great!” I honestly believe that even if you can’t consider having “a lot of classes” a virtue, you can’t call it a drawback of any truly object-oriented code either. However, there may come a point when classes become a problem; let’s see when, how, and what to do about that.

El día de la bestia (1995) by Álex de la Iglesia
El día de la bestia (1995) by Álex de la Iglesia

There were a number of “rules” previously mentioned that, if applied, would obviously lead to a large number of classes, including: a) all public methods must be declared in interfaces; b) objects must not have more than four attributes (Section 2.1 of Elegant Objects); c) static methods are not allowed; d) constructors must be code-free; e) objects must expose fewer than five public methods (Section 3.1 of Elegant Objects).

The biggest concern, of course, is maintainability: “If, instead of 50 longer classes, we had 300 shorter ones, then the code would be way less readable.” This will most certainly happen if you design them wrong.

Types (or classes) in OOP constitute your vocabulary, which explains the world around your code—the world your code lives in. The richer the vocabulary, the more powerful your code. The more types you have, the better you can understand and explain the world.

If your vocabulary is big enough, you will say something like:

Read the book that is on the table.

With a much smaller vocabulary, the same phrase would sound like:

Do it with the thing that is on that thing.

Obviously, it’s easier to read and understand the first phrase. The same occurs with types in OOP: the more of them you have at your disposal, the more expressive, bright, and readable your code is.

Unfortunately, Java and many other languages are not designed with this concept in mind. Packages, modules, and namespaces don’t really help, and we usually end up with names like AbstractCookieValueMethodArgumentResolver (Spring) or CombineFileRecordReaderWrapper (Hadoop). We’re trying to pack as many semantics into class names as possible so their users won’t doubt for a second. Then we’re trying to put as many methods into one class as possible to make life easier for users; they will use their IDE hints to find the right one.

This is anything but OOP.

If your code is object-oriented, your classes must be small, their names must be nouns, and their method names must be just one word. Here is what I do in my code to make that happen:

Interfaces are nouns. For example, Request, Directive, or Domain. There are no exceptions. Types (also known as interfaces in Java) are the core part of my vocabulary; they have to be nouns.

Classes are prefixed. My classes always implement interfaces. Thanks to that, I can say they always are requests, directives, or domains. And I always want their users to remember that. Prefixes help. For example, RqBuffered is a buffered request, RqSimple is a simple request, RqLive is a request that represents a “live” HTTP connection, and RqWithHeader is a request with an extra header.

An alternative approach is to use the type name as the central part of the class name and add a prefix that explains implementation details. For example, DyDomain is a domain that persists its data in DynamoDB. Once you know what that Dy prefix is for, you can easily understand what DyUser and DyBase are about.

In a medium-sized application or a library, there will be as many as 10 to 15 prefixes you will have to remember, no more. For example, in the Takes Framework, there are 24,000 lines of code, 410 Java files, and 10 prefixes: Bc, Cc, Tk, Rq, Rs, Fb, Fk, Hm, Ps, and Xe. Not so difficult to remember what they mean, right?

Among all 240 classes, the longest name is RqWithDefaultHeader.

I find this approach to class naming rather convenient. I used it in these open source projects (in GitHub): yegor256/takes (10 prefixes), yegor256/jare (5 prefixes), yegor256/rultor (6 prefixes), and yegor256/wring (5 prefixes).

© Yegor Bugayenko 2014–2018

Each Private Static Method Is a Candidate for a New Class

QR code

Each Private Static Method Is a Candidate for a New Class

  • Kharkiv, Ukraine
  • comments

Do you have private static methods that help you break your algorithms down into smaller parts? I do. Every time I write a new method, I realize that it can be a new class instead. Of course, I don’t make classes out of all of them, but that has to be the goal. Private static methods are not reusable, while classes are—that is the main difference between them, and it is crucial.

The Master (2012) by Paul Thomas Anderson
The Master (2012) by Paul Thomas Anderson

Here is an example of a simple class:

class Token {
  private String key;
  private String secret;
  String encoded() {
    return "key="
      + URLEncoder.encode(key, "UTF-8")
      + "&secret="
      + URLEncoder.encode(secret, "UTF-8");
  }
}

There is an obvious code duplication, right? The easiest way to resolve it is to introduce a private static method:

class Token {
  private String key;
  private String secret;
  String encoded() {
    return "key="
      + Token.encoded(key)
      + "&secret="
      + Token.encoded(secret);
  }
  private static String encoded(String text) {
    return URLEncoder.encode(text, "UTF-8");
  }
}

Looks much better now. But what will happen if we have another class that needs the exact same functionality? We will have to copy and paste this private static method encoded() into it, right?

A better alternative would be to introduce a new class Encoded that implements the functionality we want to share:

class Encoded {
  private final String raw;
  @Override
  public String toString() {
    return URLEncoder.encode(this.raw, "UTF-8");
  }
}

And then:

class Token {
  private String key;
  private String secret;
  String encoded() {
    return "key="
      + new Encoded(key)
      + "&secret="
      + new Encoded(secret);
  }
}

Now this functionality is 1) reusable, and 2) testable. We can easily use this class Encoded in many other places, and we can create a unit test for it. We were not able to do that with the private static method before.

See the point? The rule of thumb I’ve already figured for myself is that each private static method is a perfect candidate for a new class. That’s why we don’t have them at all in EO.

By the way, public static methods are a different story. They are also evil, but for different reasons.

P.S. Now I think that the reasons in this article are applicable to all private methods, not only static ones.

Do you have private static methods that help you break your algorithms down into smaller parts? I do. Every time I write a new method, I realize that it can be a new class instead. Of course, I don’t make classes out of all of them, but that has to be the goal. Private static methods are not reusable, while classes are—that is the main difference between them, and it is crucial.

The Master (2012) by Paul Thomas Anderson
The Master (2012) by Paul Thomas Anderson

Here is an example of a simple class:

class Token {
  private String key;
  private String secret;
  String encoded() {
    return "key="
      + URLEncoder.encode(key, "UTF-8")
      + "&secret="
      + URLEncoder.encode(secret, "UTF-8");
  }
}

There is an obvious code duplication, right? The easiest way to resolve it is to introduce a private static method:

class Token {
  private String key;
  private String secret;
  String encoded() {
    return "key="
      + Token.encoded(key)
      + "&secret="
      + Token.encoded(secret);
  }
  private static String encoded(String text) {
    return URLEncoder.encode(text, "UTF-8");
  }
}

Looks much better now. But what will happen if we have another class that needs the exact same functionality? We will have to copy and paste this private static method encoded() into it, right?

A better alternative would be to introduce a new class Encoded that implements the functionality we want to share:

class Encoded {
  private final String raw;
  @Override
  public String toString() {
    return URLEncoder.encode(this.raw, "UTF-8");
  }
}

And then:

class Token {
  private String key;
  private String secret;
  String encoded() {
    return "key="
      + new Encoded(key)
      + "&secret="
      + new Encoded(secret);
  }
}

Now this functionality is 1) reusable, and 2) testable. We can easily use this class Encoded in many other places, and we can create a unit test for it. We were not able to do that with the private static method before.

See the point? The rule of thumb I’ve already figured for myself is that each private static method is a perfect candidate for a new class. That’s why we don’t have them at all in EO.

By the way, public static methods are a different story. They are also evil, but for different reasons.

P.S. Now I think that the reasons in this article are applicable to all private methods, not only static ones.

© Yegor Bugayenko 2014–2018

Decorating Envelopes

QR code

Decorating Envelopes

  • Lviv, Ukraine
  • comments

Sometimes Very often I need a class that implements an interface by making an instance of another class. Sound weird? Let me show you an example. There are many classes of that kind in the Takes Framework, and they all are named like *Wrap. It’s a convenient design concept that, unfortunately, looks rather verbose in Java. It would be great to have something shorter, like in EO for example.

North by Northwest (1959) by Alfred Hitchcock
North by Northwest (1959) by Alfred Hitchcock

Take a look at RsHtml from Takes Framework. Its design looks like this (a simplified version with only one primary constructor):

class RsHtml extends RsWrap {
  RsHtml(final String text) {
    super(
      new RsWithType(
        new RsWithStatus(text, 200),
        "text/html"
      )
    );
  }
}

Now, let’s take a look at that RsWrap it extends:

public class RsWrap implements Response {
  private final Response origin;
  public RsWrap(final Response res) {
    this.origin = res;
  }
  @Override
  public final Iterable<String> head() {
    return this.origin.head();
  }
  @Override
  public final InputStream body() {
    return this.origin.body();
  }
}

As you see, this “decorator” doesn’t do anything except “just decorating.” It encapsulates another Response and passes through all method calls.

If it’s not clear yet, I’ll explain the purpose of RsHtml. Let’s say you have text and you want to create a Response:

String text = // you have it already
Response response = new RsWithType(
  new RsWithStatus(text, HttpURLConnection.HTTP_OK),
  "text/html"
);

Instead of doing this composition of decorators over and over again in many places, you use RsHtml:

String text = // you have it already
Response response = new RsHtml(text);

It is very convenient, but that RsWrap is very verbose. There are too many lines that don’t do anything special; they just forward all method calls to the encapsulated Response.

How about we introduce a new concept, “decorators,” with a new keyword, decorates:

class RsHtml decorates Response {
  RsHtml(final String text) {
    this(
      new RsWithType(
        new RsWithStatus(text, 200),
        "text/html"
      )
    )
  }
}

Then, in order to create an object, we just call:

Response response = new RsHtml(text);

We don’t have any new methods in the decorators, just constructors. The only purpose for these guys is to create other objects and encapsulate them. They are not really full-purpose objects. They only help us create other objects.

That’s why I would call them “decorating envelopes.”

This idea may look very similar to the Factory design pattern, but it doesn’t have static methods, which we are trying to avoid in object-oriented programming.

Sometimes Very often I need a class that implements an interface by making an instance of another class. Sound weird? Let me show you an example. There are many classes of that kind in the Takes Framework, and they all are named like *Wrap. It’s a convenient design concept that, unfortunately, looks rather verbose in Java. It would be great to have something shorter, like in EO for example.

North by Northwest (1959) by Alfred Hitchcock
North by Northwest (1959) by Alfred Hitchcock

Take a look at RsHtml from Takes Framework. Its design looks like this (a simplified version with only one primary constructor):

class RsHtml extends RsWrap {
  RsHtml(final String text) {
    super(
      new RsWithType(
        new RsWithStatus(text, 200),
        "text/html"
      )
    );
  }
}

Now, let’s take a look at that RsWrap it extends:

public class RsWrap implements Response {
  private final Response origin;
  public RsWrap(final Response res) {
    this.origin = res;
  }
  @Override
  public final Iterable<String> head() {
    return this.origin.head();
  }
  @Override
  public final InputStream body() {
    return this.origin.body();
  }
}

As you see, this “decorator” doesn’t do anything except “just decorating.” It encapsulates another Response and passes through all method calls.

If it’s not clear yet, I’ll explain the purpose of RsHtml. Let’s say you have text and you want to create a Response:

String text = // you have it already
Response response = new RsWithType(
  new RsWithStatus(text, HttpURLConnection.HTTP_OK),
  "text/html"
);

Instead of doing this composition of decorators over and over again in many places, you use RsHtml:

String text = // you have it already
Response response = new RsHtml(text);

It is very convenient, but that RsWrap is very verbose. There are too many lines that don’t do anything special; they just forward all method calls to the encapsulated Response.

How about we introduce a new concept, “decorators,” with a new keyword, decorates:

class RsHtml decorates Response {
  RsHtml(final String text) {
    this(
      new RsWithType(
        new RsWithStatus(text, 200),
        "text/html"
      )
    )
  }
}

Then, in order to create an object, we just call:

Response response = new RsHtml(text);

We don’t have any new methods in the decorators, just constructors. The only purpose for these guys is to create other objects and encapsulate them. They are not really full-purpose objects. They only help us create other objects.

That’s why I would call them “decorating envelopes.”

This idea may look very similar to the Factory design pattern, but it doesn’t have static methods, which we are trying to avoid in object-oriented programming.

© Yegor Bugayenko 2014–2018

Synchronized Decorators to Replace Thread-Safe Classes

QR code

Synchronized Decorators to Replace Thread-Safe Classes

  • Odessa, Ukraine
  • comments

You know what thread safety is, right? If not, there is a simple example below. All classes must be thread-safe, right? Not really. Some of them have to be thread-safe? Wrong again. I think none of them have to be thread-safe, while all of them have to provide synchronized decorators.

Aladdin (1992) by Ron Clements and John Musker
Aladdin (1992) by Ron Clements and John Musker

Let’s start with an example (it’s mutable, by the way):

class Position {
  private int number = 0;
  @Override
  public void increment() {
    int before = this.number;
    int after = before + 1;
    this.number = after;
  }
}

What do you think—is it thread-safe? This term refers to whether an object of this class will operate without mistakes when used by multiple threads at the same time. Let’s say we have two threads working with the same object, position, and calling its method increment() at exactly the same moment in time.

We expect the number integer to be equal to 2 when both threads finish up, because each of them will increment it once, right? However, most likely this won’t happen.

Let’s see what will happen. In both threads, before will equal 0 when they start. Then after will be set to 1. Then, both threads will do this.number = 1 and we will end up with 1 in number instead of the expected 2. See the problem? Classes with such a flaw in their design are not thread-safe.

The simplest and most obvious solution is to make our method synchronized. That will guarantee that no matter how many threads call it at the same time, they will all go sequentially, not in parallel: one thread after another. Of course, it will take longer, but it will prevent that mistake from happening:

class Position {
  private int number = 0;
  @Override
  public synchronized void increment() {
    int before = this.number;
    int after = before + 1;
    this.number = after;
  }
}

A class that guarantees it won’t break no matter how many threads are working with it is called thread-safe.

Now the question is: Do we have to make all classes thread-safe or only some of them? It would seem to be better to have all classes error-free, right? Why would anyone want an object that may break at some point? Well, not exactly. Remember, there is a performance concern involved; we don’t often have multiple threads, and we always want our objects to run as fast as possible. A between-threads synchronization mechanism will definitely slow us down.

I think the right approach is to have two classes. The first one is not thread-safe, while the other one is a synchronized decorator, which would look like this:

class SyncPosition implements Position {
  private final Position origin;
  SyncPosition(Position pos) {
    this.origin = pos;
  }
  @Override
  public synchronized void increment() {
    this.origin.increment();
  }
}

Now, when we need our position object to be thread-safe, we decorate it with SyncPosition:

Position position = new SyncPosition(
  new SimplePosition()
);

When we need a plain simple position, without any thread safety, we do this:

Position position = new SimplePosition();

Making class functionality both rich and thread-safe is, in my opinion, a violation of that famous single responsibility principle.

By the way, this problem is very close to the one of defensive programming and validators.

You know what thread safety is, right? If not, there is a simple example below. All classes must be thread-safe, right? Not really. Some of them have to be thread-safe? Wrong again. I think none of them have to be thread-safe, while all of them have to provide synchronized decorators.

Aladdin (1992) by Ron Clements and John Musker
Aladdin (1992) by Ron Clements and John Musker

Let’s start with an example (it’s mutable, by the way):

class Position {
  private int number = 0;
  @Override
  public void increment() {
    int before = this.number;
    int after = before + 1;
    this.number = after;
  }
}

What do you think—is it thread-safe? This term refers to whether an object of this class will operate without mistakes when used by multiple threads at the same time. Let’s say we have two threads working with the same object, position, and calling its method increment() at exactly the same moment in time.

We expect the number integer to be equal to 2 when both threads finish up, because each of them will increment it once, right? However, most likely this won’t happen.

Let’s see what will happen. In both threads, before will equal 0 when they start. Then after will be set to 1. Then, both threads will do this.number = 1 and we will end up with 1 in number instead of the expected 2. See the problem? Classes with such a flaw in their design are not thread-safe.

The simplest and most obvious solution is to make our method synchronized. That will guarantee that no matter how many threads call it at the same time, they will all go sequentially, not in parallel: one thread after another. Of course, it will take longer, but it will prevent that mistake from happening:

class Position {
  private int number = 0;
  @Override
  public synchronized void increment() {
    int before = this.number;
    int after = before + 1;
    this.number = after;
  }
}

A class that guarantees it won’t break no matter how many threads are working with it is called thread-safe.

Now the question is: Do we have to make all classes thread-safe or only some of them? It would seem to be better to have all classes error-free, right? Why would anyone want an object that may break at some point? Well, not exactly. Remember, there is a performance concern involved; we don’t often have multiple threads, and we always want our objects to run as fast as possible. A between-threads synchronization mechanism will definitely slow us down.

I think the right approach is to have two classes. The first one is not thread-safe, while the other one is a synchronized decorator, which would look like this:

class SyncPosition implements Position {
  private final Position origin;
  SyncPosition(Position pos) {
    this.origin = pos;
  }
  @Override
  public synchronized void increment() {
    this.origin.increment();
  }
}

Now, when we need our position object to be thread-safe, we decorate it with SyncPosition:

Position position = new SyncPosition(
  new SimplePosition()
);

When we need a plain simple position, without any thread safety, we do this:

Position position = new SimplePosition();

Making class functionality both rich and thread-safe is, in my opinion, a violation of that famous single responsibility principle.

By the way, this problem is very close to the one of defensive programming and validators.

© Yegor Bugayenko 2014–2018

Can Objects Be Friends?

QR code

Can Objects Be Friends?

  • Moscow, Russia
  • comments

As discussed before, proper encapsulation leads to a complete absence of “naked data.” However, the question remains: How can objects interact if they can’t exchange data? Eventually we have to expose some data in order to let other objects use it, right? Yes, that’s true. However, I guess I have a solution that keeps encapsulation in place while allowing objects to interact.

Raging Bull (1980) by Martin Scorsese
Raging Bull (1980) by Martin Scorsese

Say that this is our object:

class Temperature {
  private int t;
  public String toString() {
    return String.format("%d C", this.t);
  }
}

It represents a temperature. The only behavior it exposes is printing the temperature in Celsius. We don’t want to expose t, because that will lead to the “naked data” problem. We want to keep t secret, and that’s a good desire.

Now, we want to have the ability to print temperature in Fahrenheit. The most obvious approach would be to introduce another method, toFahrenheitString(), or add a Boolean flag to the object, which will change the behavior of method toString(), right? Either one of these solutions is better than adding a method getT(), but neither one is perfect.

What if we create this decorator:

class TempFahrenheit implements Temperature {
  private TempCelsius origin;
  public String toString() {
    return String.format(
      "%d F", this.origin.t * 1.8 + 32
    );
  }
}

It should work just great:

Temperature t = new TempFahrenheit(
  new TempCelsius(35)
);

The only problem is that it won’t compile in Java, because class TempFahrenheit is not allowed to access private t in class TempCelsius. And if we make t public, everybody will be able to read it directly, and we’ll have that “naked data” problem—a severe violation of encapsulation.

However, if we allow that access only to one class, everything will be fine. Something like this (won’t work in Java; it’s just a concept):

class TempCelsius {
  trust TempFahrenheit; // here!
  private int t;
  public String toString() {
    return String.format("%d C", this.t);
  }
}

Since this trust keyword is placed into the class that allows access, we won’t have the “naked data” problem—we will always know exactly which objects posses knowledge about t. When we change something about t, we know exactly where to update the code.

What do you think?

P.S. After discussing this idea below in comments I started to think that we don’t need that trust keyword at all. Instead, we should just give all decorators access to all private attributes of an object.

As discussed before, proper encapsulation leads to a complete absence of “naked data.” However, the question remains: How can objects interact if they can’t exchange data? Eventually we have to expose some data in order to let other objects use it, right? Yes, that’s true. However, I guess I have a solution that keeps encapsulation in place while allowing objects to interact.

Raging Bull (1980) by Martin Scorsese
Raging Bull (1980) by Martin Scorsese

Say that this is our object:

class Temperature {
  private int t;
  public String toString() {
    return String.format("%d C", this.t);
  }
}

It represents a temperature. The only behavior it exposes is printing the temperature in Celsius. We don’t want to expose t, because that will lead to the “naked data” problem. We want to keep t secret, and that’s a good desire.

Now, we want to have the ability to print temperature in Fahrenheit. The most obvious approach would be to introduce another method, toFahrenheitString(), or add a Boolean flag to the object, which will change the behavior of method toString(), right? Either one of these solutions is better than adding a method getT(), but neither one is perfect.

What if we create this decorator:

class TempFahrenheit implements Temperature {
  private TempCelsius origin;
  public String toString() {
    return String.format(
      "%d F", this.origin.t * 1.8 + 32
    );
  }
}

It should work just great:

Temperature t = new TempFahrenheit(
  new TempCelsius(35)
);

The only problem is that it won’t compile in Java, because class TempFahrenheit is not allowed to access private t in class TempCelsius. And if we make t public, everybody will be able to read it directly, and we’ll have that “naked data” problem—a severe violation of encapsulation.

However, if we allow that access only to one class, everything will be fine. Something like this (won’t work in Java; it’s just a concept):

class TempCelsius {
  trust TempFahrenheit; // here!
  private int t;
  public String toString() {
    return String.format("%d C", this.t);
  }
}

Since this trust keyword is placed into the class that allows access, we won’t have the “naked data” problem—we will always know exactly which objects posses knowledge about t. When we change something about t, we know exactly where to update the code.

What do you think?

P.S. After discussing this idea below in comments I started to think that we don’t need that trust keyword at all. Instead, we should just give all decorators access to all private attributes of an object.

© Yegor Bugayenko 2014–2018

MVC vs. OOP

QR code

MVC vs. OOP

  • Kiev, Ukraine
  • comments

Model-View-Controller (MVC) is an architectural pattern we all are well aware of. It’s a de-facto standard for almost all UI and Web frameworks. It is convenient and easy to use. It is simple and effective. It is a great concept … for a procedural programmer. If your software is object-oriented, you should dislike MVC as much as I do. Here is why.

Hot Shots! (1991) by Jim Abrahams
Hot Shots! (1991) by Jim Abrahams

This is how MVC architecture looks:

PlantUML SVG diagram

Controller is in charge, taking care of the data received from Model and injecting it into View—and this is exactly the problem. The data escapes the Model and becomes “naked,” which is a big problem, as we agreed earlier. OOP is all about encapsulation—data hiding.

MVC architecture does exactly the opposite by exposing the data and hiding behavior. The controller deals with the data directly, making decisions about its purpose and properties, while the objects, which are supposed to know everything about the data and hide it, remain anemic. That is exactly the principle any procedural architecture is built upon; the code is in charge of the data. Take this C++ code, for example:

void print_speed() { // controller
  int s = load_from_engine(); // model
  printf("The speed is %d mph", s); // view
}

The function print_speed() is the controller. It gets the data s from the model load_from_engine() and renders it via the view printf(). Only the controller knows that the data is in miles per hour. The engine returns int without any properties. The controller simply assumed that that data is in mph. If we want to create a similar controller somewhere else, we will have to make a similar assumption again and again. That’s what the “naked data” problem is about, and it leads to serious maintainability issues.

This is an object-oriented alternative to the code above (pseudo-C++):

printf(
  new PrintedSpeed( // view
    new FormattedSpeed( // controller
      new SpeedFromEngine() // model
    )
  )
);

Here, SpeedFromEngine.speed() returns speed in mph, as an integer; FormattedSpeed.speed() returns "%d mph"; and finally, PrintedSpeed.to_str() returns the full text of the message. We can call them “model, view, and controller,” but in reality they are just objects decorating each other. It’s still the same entity—the speed. But it gets more complex and intelligent by being decorated.

We don’t tear the concept of speed apart. The speed is the speed, no matter who works with it and where it is presented. It just gets new behavior from decorators. It grows, but never falls apart.

To summarize, Controller is a pure procedural component in the MVC trio, which turns Model into a passive data holder and View into a passive data renderer. The controller, the holder, the renderer … Is it really OOP?

Model-View-Controller (MVC) is an architectural pattern we all are well aware of. It’s a de-facto standard for almost all UI and Web frameworks. It is convenient and easy to use. It is simple and effective. It is a great concept … for a procedural programmer. If your software is object-oriented, you should dislike MVC as much as I do. Here is why.

Hot Shots! (1991) by Jim Abrahams
Hot Shots! (1991) by Jim Abrahams

This is how MVC architecture looks:

PlantUML SVG diagram

Controller is in charge, taking care of the data received from Model and injecting it into View—and this is exactly the problem. The data escapes the Model and becomes “naked,” which is a big problem, as we agreed earlier. OOP is all about encapsulation—data hiding.

MVC architecture does exactly the opposite by exposing the data and hiding behavior. The controller deals with the data directly, making decisions about its purpose and properties, while the objects, which are supposed to know everything about the data and hide it, remain anemic. That is exactly the principle any procedural architecture is built upon; the code is in charge of the data. Take this C++ code, for example:

void print_speed() { // controller
  int s = load_from_engine(); // model
  printf("The speed is %d mph", s); // view
}

The function print_speed() is the controller. It gets the data s from the model load_from_engine() and renders it via the view printf(). Only the controller knows that the data is in miles per hour. The engine returns int without any properties. The controller simply assumed that that data is in mph. If we want to create a similar controller somewhere else, we will have to make a similar assumption again and again. That’s what the “naked data” problem is about, and it leads to serious maintainability issues.

This is an object-oriented alternative to the code above (pseudo-C++):

printf(
  new PrintedSpeed( // view
    new FormattedSpeed( // controller
      new SpeedFromEngine() // model
    )
  )
);

Here, SpeedFromEngine.speed() returns speed in mph, as an integer; FormattedSpeed.speed() returns "%d mph"; and finally, PrintedSpeed.to_str() returns the full text of the message. We can call them “model, view, and controller,” but in reality they are just objects decorating each other. It’s still the same entity—the speed. But it gets more complex and intelligent by being decorated.

We don’t tear the concept of speed apart. The speed is the speed, no matter who works with it and where it is presented. It just gets new behavior from decorators. It grows, but never falls apart.

To summarize, Controller is a pure procedural component in the MVC trio, which turns Model into a passive data holder and View into a passive data renderer. The controller, the holder, the renderer … Is it really OOP?

© Yegor Bugayenko 2014–2018

Encapsulation Covers Up Naked Data

QR code

Encapsulation Covers Up Naked Data

  • Moscow, Russia
  • comments

Encapsulation is the core principle of object-oriented programming that makes objects solid, cohesive, trustworthy, etc. But what exactly is encapsulation? Does it only protect against access to private attributes from outside an object? I think it’s much more. Encapsulation leads to the absence of naked data on all levels and in all forms.

Borat: Cultural Learnings of America for Make Benefit Glorious Nation of Kazakhstan (2006) by Larry Charles
Borat: Cultural Learnings of America for Make Benefit Glorious Nation of Kazakhstan (2006) by Larry Charles

This is what naked data is (C code):

int t;
t = 85;
printf("The temperature is %d F", t);

Here t is the data, which is publicly accessible by the code around it. Anyone can modify it or read it.

Why is that bad? For one reason: tight and hidden coupling.

The code around t inevitably makes a lot of assumptions about the data. For example, both lines after int t decided that the temperature is in Fahrenheit. At the moment of writing, this may be true, but this assumption couples the code with the data. If tomorrow we change t to Celsius, the code won’t know about this change. That’s why I call this coupling hidden.

If we change the type of t from int to, say, double, the printf line won’t print anything after the decimal point. Again, the coupling is there, but it’s hidden. Later on, we simply won’t be able to find all the places in our code where we made these or other assumptions about t.

This will seriously affect maintainability.

And this is not a solution, as you can imagine (Java now):

class Temperature {
  private int t;
  public int getT() { return this.t; }
  public void setT(int t) { this.t = t; }
}

It looks like an object, but the data is still naked. Anyone can retrieve t from the object and decide whether it’s Fahrenheit or Celsius, whether it has digits after the dot or not, etc. This is not encapsulation yet!

The only way to encapsulate t is to make sure nobody can touch it either directly or by retrieving it from an object. How do we do that? Just stop exposing data and start exposing functionality. Here is how, for example:

class Temperature {
  private int t;
  public String toString() {
    return String.format("%d F", this.t);
  }
}

We don’t allow anyone to retrieve t anymore. All they can do is convert temperature to text. If and when we decide to change t to Celsius, we will do it just once and in one place: in the class Temperature.

If we need other functions in the future, like math operations or conversion to Celsius, we add more methods to class Temperature. But we never let anyone touch or know about t.

This idea is close to “printers instead of getters,” which we discussed earlier, though from a much wider perspective. Here I’m saying that any data elements that escape objects are naked and lead to maintainability problems.

The question is how we can work entirely without naked data, right? Eventually we have to let objects exchange data, don’t we? Yes, that’s true. But not entirely. I’ll explain that in my next post.

Encapsulation is the core principle of object-oriented programming that makes objects solid, cohesive, trustworthy, etc. But what exactly is encapsulation? Does it only protect against access to private attributes from outside an object? I think it’s much more. Encapsulation leads to the absence of naked data on all levels and in all forms.

Borat: Cultural Learnings of America for Make Benefit Glorious Nation of Kazakhstan (2006) by Larry Charles
Borat: Cultural Learnings of America for Make Benefit Glorious Nation of Kazakhstan (2006) by Larry Charles

This is what naked data is (C code):

int t;
t = 85;
printf("The temperature is %d F", t);

Here t is the data, which is publicly accessible by the code around it. Anyone can modify it or read it.

Why is that bad? For one reason: tight and hidden coupling.

The code around t inevitably makes a lot of assumptions about the data. For example, both lines after int t decided that the temperature is in Fahrenheit. At the moment of writing, this may be true, but this assumption couples the code with the data. If tomorrow we change t to Celsius, the code won’t know about this change. That’s why I call this coupling hidden.

If we change the type of t from int to, say, double, the printf line won’t print anything after the decimal point. Again, the coupling is there, but it’s hidden. Later on, we simply won’t be able to find all the places in our code where we made these or other assumptions about t.

This will seriously affect maintainability.

And this is not a solution, as you can imagine (Java now):

class Temperature {
  private int t;
  public int getT() { return this.t; }
  public void setT(int t) { this.t = t; }
}

It looks like an object, but the data is still naked. Anyone can retrieve t from the object and decide whether it’s Fahrenheit or Celsius, whether it has digits after the dot or not, etc. This is not encapsulation yet!

The only way to encapsulate t is to make sure nobody can touch it either directly or by retrieving it from an object. How do we do that? Just stop exposing data and start exposing functionality. Here is how, for example:

class Temperature {
  private int t;
  public String toString() {
    return String.format("%d F", this.t);
  }
}

We don’t allow anyone to retrieve t anymore. All they can do is convert temperature to text. If and when we decide to change t to Celsius, we will do it just once and in one place: in the class Temperature.

If we need other functions in the future, like math operations or conversion to Celsius, we add more methods to class Temperature. But we never let anyone touch or know about t.

This idea is close to “printers instead of getters,” which we discussed earlier, though from a much wider perspective. Here I’m saying that any data elements that escape objects are naked and lead to maintainability problems.

The question is how we can work entirely without naked data, right? Eventually we have to let objects exchange data, don’t we? Yes, that’s true. But not entirely. I’ll explain that in my next post.

© Yegor Bugayenko 2014–2018

Don't Use Java Assertions

QR code

Don't Use Java Assertions

  • Los Angeles, CA
  • comments

There are basically two ways to validate a situation in Java and complain when something unexpected happens. It’s either an exception or an assertion. Technically, they are almost the same, but there are some small differences. I believe that exceptions are the right way to go in all situations and assertions should never be used. Here’s why.

Natural Born Killers (1994) by Oliver Stone
Natural Born Killers (1994) by Oliver Stone

Let’s see what happens when an assertion is triggered. Say that this is our code:

public class Main {
  public static void main(String... args) {
    assert true == false : "There is a problem";
    System.out.println("Hello, world!");
  }
}

Save this code to Main.java and compile:

$ javac Main.java

Then run it:

$ java Main
Hello, world!

The assertion wasn’t triggered. It was ignored. Now run it with an -enableassertions flag:

$ java -enableassertions Main
Exception in thread "main" java.lang.AssertionError: There is a problem
  at Main.main(Main.java:3)

This is the first difference between exceptions and assertions. Exceptions will always be thrown, while assertions are not enabled by default. They are supposed to be turned on during testing and turned off in production. The assertion caused the runtime exception AssertionError. But hold on; it’s not a RuntimeException. It extends Error class, which extends Throwable. This is the second difference. I don’t know of any other differences.

I would recommend not to use assertions … ever. Simply because I strongly believe in the Fail Fast approach. I think bugs must be visible not only during testing but also in production. Moreover, I believe making bugs visible in production is very important if you want to achieve a high-quality product.

Thus, no assertions. They are simply a flawed and outdated feature in Java (and some other languages).

There are basically two ways to validate a situation in Java and complain when something unexpected happens. It’s either an exception or an assertion. Technically, they are almost the same, but there are some small differences. I believe that exceptions are the right way to go in all situations and assertions should never be used. Here’s why.

Natural Born Killers (1994) by Oliver Stone
Natural Born Killers (1994) by Oliver Stone

Let’s see what happens when an assertion is triggered. Say that this is our code:

public class Main {
  public static void main(String... args) {
    assert true == false : "There is a problem";
    System.out.println("Hello, world!");
  }
}

Save this code to Main.java and compile:

$ javac Main.java

Then run it:

$ java Main
Hello, world!

The assertion wasn’t triggered. It was ignored. Now run it with an -enableassertions flag:

$ java -enableassertions Main
Exception in thread "main" java.lang.AssertionError: There is a problem
  at Main.main(Main.java:3)

This is the first difference between exceptions and assertions. Exceptions will always be thrown, while assertions are not enabled by default. They are supposed to be turned on during testing and turned off in production. The assertion caused the runtime exception AssertionError. But hold on; it’s not a RuntimeException. It extends Error class, which extends Throwable. This is the second difference. I don’t know of any other differences.

I would recommend not to use assertions … ever. Simply because I strongly believe in the Fail Fast approach. I think bugs must be visible not only during testing but also in production. Moreover, I believe making bugs visible in production is very important if you want to achieve a high-quality product.

Thus, no assertions. They are simply a flawed and outdated feature in Java (and some other languages).

© Yegor Bugayenko 2014–2018

Test Methods Must Share Nothing

QR code

Test Methods Must Share Nothing

  • Palo Alto, CA
  • comments

Constants… I wrote about them some time ago, mostly saying that they are a bad thing, if being public. They reduce duplication, but introduce coupling. A much better way to get rid of duplication is by creating new classes or methods—a traditional OOP method. This seems to make sense and in our projects I see less and less public constants. In some projects we don’t have them at all. But one thing still bothers me: unit tests. Most programmers seem to think that when static analysis says that there are too many similar literals in the same file, the best way to get rid of them is via a private static literal. This is just wrong.

Oldeuboi (2003) by Chan-wook Park
Oldeuboi (2003) by Chan-wook Park

Unit tests, naturally, duplicate a lot of code. Test methods contain similar or almost identical functionality and this is almost inevitable. Well, we can use more of that @Before and @BeforeClass features, but sometimes it’s just not possible. We may have, say, 20 test methods in one FooTest.java file. Preparing all objects in one “before” is not possible. So we have to do certain things again and again in our test methods.

Let’s take a look at one of the classes in our Takes Framework: VerboseListTest. It’s a unit test and it has a problem, which I’m trying to tell you about. Look at that MSG private literal. It is used for the first time in setUp() method as an argument of an object constructor and then in a few test methods to check how that object behaves. Let me simplify that code:

class FooTest {
  private static final String MSG = "something";
  @Before
  public final void setUp() throws Exception {
    this.foo = new Foo(FooTest.MSG);
  }
  @Test
  public void simplyWorks() throws IOException {
    assertThat(
      foo.doSomething(),
      containsString(FooTest.MSG)
    );
  }
  @Test
  public void simplyWorksAgain() throws IOException {
    assertThat(
      foo.doSomethingElse(),
      containsString(FooTest.MSG)
    );
  }
}

This is basically what is happening in VerboseListTest and it’s very wrong. Why? Because this shared literal MSG introduced an unnatural coupling between these two test methods. They have nothing in common, because they test different behaviors of class Foo. But this private constant ties them together. Now they are somehow related.

If and when I want to modify one of the test methods, I may need to modify the other one too. Say I want to see how doSomethingElse() behaves if the encapsulated message is an empty string. What do I do? I change the value of the constant FooTest.MSG, which is used by another test method. This is called coupling. And it’s a bad thing.

What do we do? Well, we can use that "something" string literal in both test methods:

class FooTest {
  @Test
  public void simplyWorks() throws IOException {
    assertThat(
      new Foo("something").doSomething(),
      containsString("something")
    );
  }
  @Test
  public void simplyWorksAgain() throws IOException {
    assertThat(
      new Foo("something").doSomethingElse(),
      containsString("something")
    );
  }
}

As you see, I got rid of that setUp() method and the private static literal MSG. What do we have now? Code duplication. String "something" shows up four times in the test class. No static analyzers will tolerate that. Moreover, there are seven (!) test methods in VerboseListTest, which are using MSG. Thus, we will have 14 occurrences of "something", right? Yes, that’s right and that’s most likely why one of authors of this test case introduced the constant—to get rid of duplication. BTW, @Happy-Neko did that in pull request #513, @carlosmiranda reviewed the code and I approved the changes. So, three people made/approved that mistake, including myself.

So what is the right approach that will avoid code duplication and at the same time won’t introduce coupling? Here it is:

class FooTest {
  @Test
  public void simplyWorks() throws IOException {
    final String msg = "something";
    assertThat(
      new Foo(msg).doSomething(),
      containsString(msg)
    );
  }
  @Test
  public void simplyWorksAgain() throws IOException {
    final String msg = "something else";
    assertThat(
      new Foo(msg).doSomethingElse(),
      containsString(msg)
    );
  }
}

These literals must be different. This is what any static analyzer is saying when it sees "something" in so many places. It questions us—why are they the same? Is it really so important to use "something" everywhere? Why can’t you use different literals? Of course we can. And we should.

The bottom line is that each test method must have its own set of data and objects. They must not be shared between test methods ever. Test methods must always be independent, having nothing in common.

Having that in mind, we can easily conclude that methods like setUp() or any shared variables in test classes are evil. They must not be used and simply must not exist. I think that their invention in JUnit caused a lot of harm to Java code.

Constants… I wrote about them some time ago, mostly saying that they are a bad thing, if being public. They reduce duplication, but introduce coupling. A much better way to get rid of duplication is by creating new classes or methods—a traditional OOP method. This seems to make sense and in our projects I see less and less public constants. In some projects we don’t have them at all. But one thing still bothers me: unit tests. Most programmers seem to think that when static analysis says that there are too many similar literals in the same file, the best way to get rid of them is via a private static literal. This is just wrong.

Oldeuboi (2003) by Chan-wook Park
Oldeuboi (2003) by Chan-wook Park

Unit tests, naturally, duplicate a lot of code. Test methods contain similar or almost identical functionality and this is almost inevitable. Well, we can use more of that @Before and @BeforeClass features, but sometimes it’s just not possible. We may have, say, 20 test methods in one FooTest.java file. Preparing all objects in one “before” is not possible. So we have to do certain things again and again in our test methods.

Let’s take a look at one of the classes in our Takes Framework: VerboseListTest. It’s a unit test and it has a problem, which I’m trying to tell you about. Look at that MSG private literal. It is used for the first time in setUp() method as an argument of an object constructor and then in a few test methods to check how that object behaves. Let me simplify that code:

class FooTest {
  private static final String MSG = "something";
  @Before
  public final void setUp() throws Exception {
    this.foo = new Foo(FooTest.MSG);
  }
  @Test
  public void simplyWorks() throws IOException {
    assertThat(
      foo.doSomething(),
      containsString(FooTest.MSG)
    );
  }
  @Test
  public void simplyWorksAgain() throws IOException {
    assertThat(
      foo.doSomethingElse(),
      containsString(FooTest.MSG)
    );
  }
}

This is basically what is happening in VerboseListTest and it’s very wrong. Why? Because this shared literal MSG introduced an unnatural coupling between these two test methods. They have nothing in common, because they test different behaviors of class Foo. But this private constant ties them together. Now they are somehow related.

If and when I want to modify one of the test methods, I may need to modify the other one too. Say I want to see how doSomethingElse() behaves if the encapsulated message is an empty string. What do I do? I change the value of the constant FooTest.MSG, which is used by another test method. This is called coupling. And it’s a bad thing.

What do we do? Well, we can use that "something" string literal in both test methods:

class FooTest {
  @Test
  public void simplyWorks() throws IOException {
    assertThat(
      new Foo("something").doSomething(),
      containsString("something")
    );
  }
  @Test
  public void simplyWorksAgain() throws IOException {
    assertThat(
      new Foo("something").doSomethingElse(),
      containsString("something")
    );
  }
}

As you see, I got rid of that setUp() method and the private static literal MSG. What do we have now? Code duplication. String "something" shows up four times in the test class. No static analyzers will tolerate that. Moreover, there are seven (!) test methods in VerboseListTest, which are using MSG. Thus, we will have 14 occurrences of "something", right? Yes, that’s right and that’s most likely why one of authors of this test case introduced the constant—to get rid of duplication. BTW, @Happy-Neko did that in pull request #513, @carlosmiranda reviewed the code and I approved the changes. So, three people made/approved that mistake, including myself.

So what is the right approach that will avoid code duplication and at the same time won’t introduce coupling? Here it is:

class FooTest {
  @Test
  public void simplyWorks() throws IOException {
    final String msg = "something";
    assertThat(
      new Foo(msg).doSomething(),
      containsString(msg)
    );
  }
  @Test
  public void simplyWorksAgain() throws IOException {
    final String msg = "something else";
    assertThat(
      new Foo(msg).doSomethingElse(),
      containsString(msg)
    );
  }
}

These literals must be different. This is what any static analyzer is saying when it sees "something" in so many places. It questions us—why are they the same? Is it really so important to use "something" everywhere? Why can’t you use different literals? Of course we can. And we should.

The bottom line is that each test method must have its own set of data and objects. They must not be shared between test methods ever. Test methods must always be independent, having nothing in common.

Having that in mind, we can easily conclude that methods like setUp() or any shared variables in test classes are evil. They must not be used and simply must not exist. I think that their invention in JUnit caused a lot of harm to Java code.

© Yegor Bugayenko 2014–2018

Why InputStream Design Is Wrong

QR code

Why InputStream Design Is Wrong

  • Washington, D.C.
  • comments

It’s not just about InputSteam, this class is a good example of a bad design. I’m talking about three overloaded methods read(). I’ve mentioned this problem in Section 2.9 of Elegant Objects. In a few words, I strongly believe that interfaces must be “functionality poor.” InputStream should have been an interface in the first place and it should have had a single method read(byte[]). Then if its authors wanted to give us extra functionality, they should have created supplementary “smart” classes.

A Serious Man (2009) by Coen Brothers
A Serious Man (2009) by Coen Brothers

This is how it looks now:

abstract class InputStream {
  int read();
  int read(byte[] buffer, int offset, int length);
  int read(byte[] buffer);
}

What’s wrong? It’s very convenient to have the ability to read a single byte, an array of bytes or even an array of bytes with a direct positioning into a specific place in the buffer!

However, we are still lacking a few methods: for reading the bytes and immediately saving into a file, converting to a text with a selected encoding, sending them by email and posting on Twitter. It would be great to have the features too, right in the poor InputStream. I hope the Oracle Java team is working on them now.

In the mean time, let’s see what exactly is wrong with what these bright engineers designed for us already. Or maybe let me show how I would design InputStream and we’ll compare:

interface InputStream {
  int read(byte[] buffer, int offset, int length);
}

This is my design. The InputStream is responsible for reading bytes from the stream. There is one single method for this feature. Is it convenient for everybody? Does it read and post on Twitter? Not yet. Do we need that functionality? Of course we do, but it doesn’t mean that we will add it to the interface. Instead, we will create supplementary “smart” class:

interface InputStream {
  int read(byte[] buffer, int offset, int length);
  class Smart {
    private final InputStream origin;
    public Smart(InputStream stream) {
      this.origin = stream;
    }
    public int read() {
      final byte[] buffer = new byte[1];
      final int read = this.origin.read(buffer, 0, 1);
      final int result;
      if (read < 1) {
        result = -1;
      } else {
        result = buffer[0];
      }
      return result;
    }
  }
}

Now, we want to read a single byte from the stream. Here is how:

final InputStream input = new FileInputStream("/tmp/a.txt");
final byte b = new InputStream.Smart(input).read();

The functionality of reading a single byte is outside of InputStream, because this is not its business. The stream doesn’t need to know how to manage the data after it is read. All the stream is responsible for is reading, not parsing or manipulating afterwards.

Interfaces must be small.

Obviously, method overloading in interfaces is a code smell. An interface with more than three methods is a good candidate for refactoring. If methods overload each other—it’s serious trouble.

Interfaces must be small!

You may say that the creators of InputStream cared about performance, that’s why allowed us to implement read() in three different forms. Then I have to ask again, why not create a method for reading and immediately post it on Twitter? That would be fantastically fast. Isn’t it what we all want? A fast software which nobody has any desire to read or maintain.

It’s not just about InputSteam, this class is a good example of a bad design. I’m talking about three overloaded methods read(). I’ve mentioned this problem in Section 2.9 of Elegant Objects. In a few words, I strongly believe that interfaces must be “functionality poor.” InputStream should have been an interface in the first place and it should have had a single method read(byte[]). Then if its authors wanted to give us extra functionality, they should have created supplementary “smart” classes.

A Serious Man (2009) by Coen Brothers
A Serious Man (2009) by Coen Brothers

This is how it looks now:

abstract class InputStream {
  int read();
  int read(byte[] buffer, int offset, int length);
  int read(byte[] buffer);
}

What’s wrong? It’s very convenient to have the ability to read a single byte, an array of bytes or even an array of bytes with a direct positioning into a specific place in the buffer!

However, we are still lacking a few methods: for reading the bytes and immediately saving into a file, converting to a text with a selected encoding, sending them by email and posting on Twitter. It would be great to have the features too, right in the poor InputStream. I hope the Oracle Java team is working on them now.

In the mean time, let’s see what exactly is wrong with what these bright engineers designed for us already. Or maybe let me show how I would design InputStream and we’ll compare:

interface InputStream {
  int read(byte[] buffer, int offset, int length);
}

This is my design. The InputStream is responsible for reading bytes from the stream. There is one single method for this feature. Is it convenient for everybody? Does it read and post on Twitter? Not yet. Do we need that functionality? Of course we do, but it doesn’t mean that we will add it to the interface. Instead, we will create supplementary “smart” class:

interface InputStream {
  int read(byte[] buffer, int offset, int length);
  class Smart {
    private final InputStream origin;
    public Smart(InputStream stream) {
      this.origin = stream;
    }
    public int read() {
      final byte[] buffer = new byte[1];
      final int read = this.origin.read(buffer, 0, 1);
      final int result;
      if (read < 1) {
        result = -1;
      } else {
        result = buffer[0];
      }
      return result;
    }
  }
}

Now, we want to read a single byte from the stream. Here is how:

final InputStream input = new FileInputStream("/tmp/a.txt");
final byte b = new InputStream.Smart(input).read();

The functionality of reading a single byte is outside of InputStream, because this is not its business. The stream doesn’t need to know how to manage the data after it is read. All the stream is responsible for is reading, not parsing or manipulating afterwards.

Interfaces must be small.

Obviously, method overloading in interfaces is a code smell. An interface with more than three methods is a good candidate for refactoring. If methods overload each other—it’s serious trouble.

Interfaces must be small!

You may say that the creators of InputStream cared about performance, that’s why allowed us to implement read() in three different forms. Then I have to ask again, why not create a method for reading and immediately post it on Twitter? That would be fantastically fast. Isn’t it what we all want? A fast software which nobody has any desire to read or maintain.

© Yegor Bugayenko 2014–2018

Object Behavior Must Not Be Configurable

QR code

Object Behavior Must Not Be Configurable

  • New York, NY
  • comments

Using object properties as configuration parameters is a very common mistake we keep making mostly because our objects are mutable—we configure them. We change their behavior by injecting parameters or even entire settings/configuration objects into them. Do I have to say that it’s abusive and disrespectful from a philosophical point of view? I can, but let’s take a look at it from a practical perspective.

The Take (2009) by David Drury
The Take (2009) by David Drury

Let’s say there is a class that is supposed to read a web page and return its content:

class Page {
  private final String uri;
  Page(final String address) {
    this.uri = address;
  }
  public String html() throws IOException {
    return IOUtils.toString(
      new URL(this.uri).openStream(),
      "UTF-8"
    );
  }
}

Looks simple and straight-forward, right? Yes, it’s a rather cohesive and solid class. Here is how we use it to read the content of Google front page:

String html = new Page("http://www.google.com").html();

Everything is fine until we start making this class more powerful. Let’s say we want to configure the encoding. We don’t always want to use "UTF-8". We want it to be configurable. Here is what we do:

class Page {
  private final String uri;
  private final String encoding;
  Page(final String address, final String enc) {
    this.uri = address;
    this.encoding = enc;
  }
  public String html() throws IOException {
    return IOUtils.toString(
      new URL(this.uri).openStream(),
      this.encoding
    );
  }
}

Done, the encoding is encapsulated and configurable. Now, let’s say we want to change the behavior of the class for the situation of an empty page. If an empty page is loaded, we want to return "<html/>". But not always. We want this to be configurable. Here is what we do:

class Page {
  private final String uri;
  private final String encoding;
  private final boolean alwaysHtml;
  Page(final String address, final String enc,
    final boolean always) {
    this.uri = address;
    this.encoding = enc;
    this.alwaysHtml = always;
  }
  public String html() throws IOException {
    String html = IOUtils.toString(
      new URL(this.uri).openStream(),
      this.encoding
    );
    if (html.isEmpty() && this.alwaysHtml) {
      html = "<html/>";
    }
    return html;
  }
}

The class is getting bigger, huh? It’s great, we’re good programmers and our code must be complex, right? The more complex it is, the better programmers we are! I’m being sarcastic. Definitely not! But let’s move on. Now we want our class to proceed anyway, even if the encoding is not supported on the current platform:

class Page {
  private final String uri;
  private final String encoding;
  private final boolean alwaysHtml;
  private final boolean encodeAnyway;
  Page(final String address, final String enc,
    final boolean always, final boolean encode) {
    this.uri = address;
    this.encoding = enc;
    this.alwaysHtml = always;
    this.encodeAnyway = encode;
  }
  public String html() throws IOException,
  UnsupportedEncodingException {
    final byte[] bytes = IOUtils.toByteArray(
      new URL(this.uri).openStream()
    );
    String html;
    try {
      html = new String(bytes, this.encoding);
    } catch (UnsupportedEncodingException ex) {
      if (!this.encodeAnyway) {
        throw ex;
      }
      html = new String(bytes, "UTF-8")
    }
    if (html.isEmpty() && this.alwaysHtml) {
      html = "<html/>";
    }
    return html;
  }
}

The class is growing and becoming more and more powerful! Now it’s time to introduce a new class, which we will call PageSettings:

class Page {
  private final String uri;
  private final PageSettings settings;
  Page(final String address, final PageSettings stts) {
    this.uri = address;
    this.settings = stts;
  }
  public String html() throws IOException {
    final byte[] bytes = IOUtils.toByteArray(
      new URL(this.uri).openStream()
    );
    String html;
    try {
      html = new String(bytes, this.settings.getEncoding());
    } catch (UnsupportedEncodingException ex) {
      if (!this.settings.isEncodeAnyway()) {
        throw ex;
      }
      html = new String(bytes, "UTF-8")
    }
    if (html.isEmpty() && this.settings.isAlwaysHtml()) {
      html = "<html/>";
    }
    return html;
  }
}

Class PageSettings is basically a holder of parameters, without any behavior. It has getters, which give us access to the parameters: isEncodeAnyway(), isAlwaysHtml(), and getEncoding(). If we keep going in this direction, there could be a few dozen configuration settings in that class. This may look very convenient and is a very typical pattern in Java world. For example, look at JobConf from Hadoop. This is how we will call our highly configurable Page (I’m assuming PageSettings is immutable):

String html = new Page(
  "http://www.google.com",
  new PageSettings()
    .withEncoding("ISO_8859_1")
    .withAlwaysHtml(true)
    .withEncodeAnyway(false)
).html();

However, no matter how convenient it may look at first glance, this approach is very wrong. Mostly because it encourages us to make big and non-cohesive objects. They grow in size and become less testable, less maintainable and less readable.

To prevent that from happening, I would suggest a simple rule here: object behavior should not be configurable. Or, more technically, encapsulated properties must not be used to change the behavior of an object.

Object properties are there only to coordinate the location of a real-world entity, which the object is representing. The uri is the coordinate, while the alwaysHtml boolean property is a behavior changing trigger. See the difference?

So, what should we do instead? What is the right design? We must use composable decorators. Here is how:

Page page = new NeverEmptyPage(
  new DefaultPage("http://www.google.com")
)
String html = new AlwaysTextPage(
  new TextPage(page, "ISO_8859_1")
  page
).html();

Here is how our DefaultPage would look (yes, I had to change its design a bit):

class DefaultPage implements Page {
  private final String uri;
  DefaultPage(final String address) {
    this.uri = address;
  }
  @Override
  public byte[] html() throws IOException {
    return IOUtils.toByteArray(
      new URL(this.uri).openStream()
    );
  }
}

As you see, I’m making it implement interface Page. Now TextPage decorator, which converts an array of bytes to a text using provided encoding:

class TextPage {
  private final Page origin;
  private final String encoding;
  TextPage(final Page page, final String enc) {
    this.origin = page;
    this.encoding = enc;
  }
  public String html() throws IOException {
    return new String(
      this.origin.html(),
      this.encoding
    );
  }
}

Now the NeverEmptyPage:

class NeverEmptyPage implements Page {
  private final Page origin;
  NeverEmptyPage(final Page page) {
    this.origin = page;
  }
  @Override
  public byte[] html() throws IOException {
    byte[] bytes = this.origin.html();
    if (bytes.length == 0) {
      bytes = "<html/>".getBytes();
    }
    return bytes;
  }
}

And finally the AlwaysTextPage:

class AlwaysTextPage {
  private final TextPage origin;
  private final Page source;
  AlwaysTextPage(final TextPage page, final Page src) {
    this.origin = page;
    this.source = src;
  }
  public String html() throws IOException {
    String html;
    try {
      html = this.origin.html();
    } catch (UnsupportedEncodingException ex) {
      html = new TextPage(this.source, "UTF-8").html();
    }
    return html;
  }
}

You may say that AlwaysTextPage will make two calls to the encapsulated origin, in case of an unsupported encoding, which will lead to a duplicated HTTP request. That’s true and this is by design. We don’t want this duplicated HTTP roundtrip to happen. Let’s introduce one more class, which will cache the page fetched ( not thread-safe, but it’s not important now):

class OncePage implements Page {
  private final Page origin;
  private final AtomicReference<byte[]> cache =
    new AtomicReference<>;
  OncePage(final Page page) {
    this.origin = page;
  }
  @Override
  public byte[] html() throws IOException {
    if (this.cache.get() == null) {
      this.cache.set(this.origin.html());
    }
    return this.cache.get();
  }
}

Now, our code should look like this (pay attention, I’m now using OncePage):

Page page = new NeverEmptyPage(
  new OncePage(
    new DefaultPage("http://www.google.com")
  )
)
String html = new AlwaysTextPage(
  new TextPage(page, "ISO_8859_1")
  "UTF-8"
).html();

This is probably the most code-intensive post on this site so far, but I hope it’s readable and I managed to convey the idea. Now we have five classes, each of which is rather small, easy to read and easy to reuse.

Just follow the rule: never make classes configurable!

Using object properties as configuration parameters is a very common mistake we keep making mostly because our objects are mutable—we configure them. We change their behavior by injecting parameters or even entire settings/configuration objects into them. Do I have to say that it’s abusive and disrespectful from a philosophical point of view? I can, but let’s take a look at it from a practical perspective.

The Take (2009) by David Drury
The Take (2009) by David Drury

Let’s say there is a class that is supposed to read a web page and return its content:

class Page {
  private final String uri;
  Page(final String address) {
    this.uri = address;
  }
  public String html() throws IOException {
    return IOUtils.toString(
      new URL(this.uri).openStream(),
      "UTF-8"
    );
  }
}

Looks simple and straight-forward, right? Yes, it’s a rather cohesive and solid class. Here is how we use it to read the content of Google front page:

String html = new Page("http://www.google.com").html();

Everything is fine until we start making this class more powerful. Let’s say we want to configure the encoding. We don’t always want to use "UTF-8". We want it to be configurable. Here is what we do:

class Page {
  private final String uri;
  private final String encoding;
  Page(final String address, final String enc) {
    this.uri = address;
    this.encoding = enc;
  }
  public String html() throws IOException {
    return IOUtils.toString(
      new URL(this.uri).openStream(),
      this.encoding
    );
  }
}

Done, the encoding is encapsulated and configurable. Now, let’s say we want to change the behavior of the class for the situation of an empty page. If an empty page is loaded, we want to return "<html/>". But not always. We want this to be configurable. Here is what we do:

class Page {
  private final String uri;
  private final String encoding;
  private final boolean alwaysHtml;
  Page(final String address, final String enc,
    final boolean always) {
    this.uri = address;
    this.encoding = enc;
    this.alwaysHtml = always;
  }
  public String html() throws IOException {
    String html = IOUtils.toString(
      new URL(this.uri).openStream(),
      this.encoding
    );
    if (html.isEmpty() && this.alwaysHtml) {
      html = "<html/>";
    }
    return html;
  }
}

The class is getting bigger, huh? It’s great, we’re good programmers and our code must be complex, right? The more complex it is, the better programmers we are! I’m being sarcastic. Definitely not! But let’s move on. Now we want our class to proceed anyway, even if the encoding is not supported on the current platform:

class Page {
  private final String uri;
  private final String encoding;
  private final boolean alwaysHtml;
  private final boolean encodeAnyway;
  Page(final String address, final String enc,
    final boolean always, final boolean encode) {
    this.uri = address;
    this.encoding = enc;
    this.alwaysHtml = always;
    this.encodeAnyway = encode;
  }
  public String html() throws IOException,
  UnsupportedEncodingException {
    final byte[] bytes = IOUtils.toByteArray(
      new URL(this.uri).openStream()
    );
    String html;
    try {
      html = new String(bytes, this.encoding);
    } catch (UnsupportedEncodingException ex) {
      if (!this.encodeAnyway) {
        throw ex;
      }
      html = new String(bytes, "UTF-8")
    }
    if (html.isEmpty() && this.alwaysHtml) {
      html = "<html/>";
    }
    return html;
  }
}

The class is growing and becoming more and more powerful! Now it’s time to introduce a new class, which we will call PageSettings:

class Page {
  private final String uri;
  private final PageSettings settings;
  Page(final String address, final PageSettings stts) {
    this.uri = address;
    this.settings = stts;
  }
  public String html() throws IOException {
    final byte[] bytes = IOUtils.toByteArray(
      new URL(this.uri).openStream()
    );
    String html;
    try {
      html = new String(bytes, this.settings.getEncoding());
    } catch (UnsupportedEncodingException ex) {
      if (!this.settings.isEncodeAnyway()) {
        throw ex;
      }
      html = new String(bytes, "UTF-8")
    }
    if (html.isEmpty() && this.settings.isAlwaysHtml()) {
      html = "<html/>";
    }
    return html;
  }
}

Class PageSettings is basically a holder of parameters, without any behavior. It has getters, which give us access to the parameters: isEncodeAnyway(), isAlwaysHtml(), and getEncoding(). If we keep going in this direction, there could be a few dozen configuration settings in that class. This may look very convenient and is a very typical pattern in Java world. For example, look at JobConf from Hadoop. This is how we will call our highly configurable Page (I’m assuming PageSettings is immutable):

String html = new Page(
  "http://www.google.com",
  new PageSettings()
    .withEncoding("ISO_8859_1")
    .withAlwaysHtml(true)
    .withEncodeAnyway(false)
).html();

However, no matter how convenient it may look at first glance, this approach is very wrong. Mostly because it encourages us to make big and non-cohesive objects. They grow in size and become less testable, less maintainable and less readable.

To prevent that from happening, I would suggest a simple rule here: object behavior should not be configurable. Or, more technically, encapsulated properties must not be used to change the behavior of an object.

Object properties are there only to coordinate the location of a real-world entity, which the object is representing. The uri is the coordinate, while the alwaysHtml boolean property is a behavior changing trigger. See the difference?

So, what should we do instead? What is the right design? We must use composable decorators. Here is how:

Page page = new NeverEmptyPage(
  new DefaultPage("http://www.google.com")
)
String html = new AlwaysTextPage(
  new TextPage(page, "ISO_8859_1")
  page
).html();

Here is how our DefaultPage would look (yes, I had to change its design a bit):

class DefaultPage implements Page {
  private final String uri;
  DefaultPage(final String address) {
    this.uri = address;
  }
  @Override
  public byte[] html() throws IOException {
    return IOUtils.toByteArray(
      new URL(this.uri).openStream()
    );
  }
}

As you see, I’m making it implement interface Page. Now TextPage decorator, which converts an array of bytes to a text using provided encoding:

class TextPage {
  private final Page origin;
  private final String encoding;
  TextPage(final Page page, final String enc) {
    this.origin = page;
    this.encoding = enc;
  }
  public String html() throws IOException {
    return new String(
      this.origin.html(),
      this.encoding
    );
  }
}

Now the NeverEmptyPage:

class NeverEmptyPage implements Page {
  private final Page origin;
  NeverEmptyPage(final Page page) {
    this.origin = page;
  }
  @Override
  public byte[] html() throws IOException {
    byte[] bytes = this.origin.html();
    if (bytes.length == 0) {
      bytes = "<html/>".getBytes();
    }
    return bytes;
  }
}

And finally the AlwaysTextPage:

class AlwaysTextPage {
  private final TextPage origin;
  private final Page source;
  AlwaysTextPage(final TextPage page, final Page src) {
    this.origin = page;
    this.source = src;
  }
  public String html() throws IOException {
    String html;
    try {
      html = this.origin.html();
    } catch (UnsupportedEncodingException ex) {
      html = new TextPage(this.source, "UTF-8").html();
    }
    return html;
  }
}

You may say that AlwaysTextPage will make two calls to the encapsulated origin, in case of an unsupported encoding, which will lead to a duplicated HTTP request. That’s true and this is by design. We don’t want this duplicated HTTP roundtrip to happen. Let’s introduce one more class, which will cache the page fetched ( not thread-safe, but it’s not important now):

class OncePage implements Page {
  private final Page origin;
  private final AtomicReference<byte[]> cache =
    new AtomicReference<>;
  OncePage(final Page page) {
    this.origin = page;
  }
  @Override
  public byte[] html() throws IOException {
    if (this.cache.get() == null) {
      this.cache.set(this.origin.html());
    }
    return this.cache.get();
  }
}

Now, our code should look like this (pay attention, I’m now using OncePage):

Page page = new NeverEmptyPage(
  new OncePage(
    new DefaultPage("http://www.google.com")
  )
)
String html = new AlwaysTextPage(
  new TextPage(page, "ISO_8859_1")
  "UTF-8"
).html();

This is probably the most code-intensive post on this site so far, but I hope it’s readable and I managed to convey the idea. Now we have five classes, each of which is rather small, easy to read and easy to reuse.

Just follow the rule: never make classes configurable!

© Yegor Bugayenko 2014–2018

Java Annotations Are a Big Mistake

QR code

Java Annotations Are a Big Mistake

  • Seattle, WA
  • comments

Annotations were introduced in Java 5, and we all got excited. Such a great instrument to make code shorter! No more Hibernate/Spring XML configuration files! Just annotations, right there in the code where we need them. No more marker interfaces, just a runtime-retained reflection-discoverable annotation! I was excited too. Moreover, I’ve made a few open source libraries which use annotations heavily. Take jcabi-aspects, for example. However, I’m not excited any more. Moreover, I believe that annotations are a big mistake in Java design.

Gomorra (2008) by Matteo Garrone)
Gomorra (2008) by Matteo Garrone)

Long story short, there is one big problem with annotations—they encourage us to implement object functionality outside of an object, which is against the very principle of encapsulation. The object is not solid any more, since its behavior is not defined entirely by its own methods—some of its functionality stays elsewhere. Why is it bad? Let’s see in a few examples.

@Inject

Say we annotate a property with @Inject:

import javax.inject.Inject;
public class Books {
  @Inject
  private final DB db;
  // some methods here, which use this.db
}

Then we have an injector that knows what to inject:

Injector injector = Guice.createInjector(
  new AbstractModule() {
    @Override
    public void configure() {
      this.bind(DB.class).toInstance(
        new Postgres("jdbc:postgresql:5740/main")
      );
    }
  }
);

Now we’re making an instance of class Books via the container:

Books books = injector.getInstance(Books.class);

The class Books has no idea how and who will inject an instance of class DB into it. This will happen behind the scenes and outside of its control. The injection will do it. It may look convenient, but this attitude causes a lot of damage to the entire code base. The control is lost (not inverted, but lost!). The object is not in charge any more. It can’t be responsible for what’s happening to it.

Instead, here is how this should be done:

class Books {
  private final DB db;
  Books(final DB base) {
    this.db = base;
  }
  // some methods here, which use this.db
}

This article explains why Dependency Injection containers are a wrong idea in the first place: Dependency Injection Containers are Code Polluters. Annotations basically provoke us to make the containers and use them. We move functionality outside of our objects and put it into containers, or somewhere else. That’s because we don’t want to duplicate the same code over and over again, right? That’s correct, duplication is bad, but tearing an object apart is even worse. Way worse. The same is true about ORM (JPA/Hibernate), where annotations are being actively used. Check this post, it explains what is wrong about ORM: ORM Is an Offensive Anti-Pattern. Annotations by themselves are not the key motivator, but they help us and encourage us by tearing objects apart and keeping parts in different places. They are containers, sessions, managers, controllers, etc.

@XmlElement

This is how JAXB works, when you want to convert your POJO to XML. First, you attach the @XmlElement annotation to the getter:

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
@XmlRootElement
public class Book {
  private final String title;
  public Book(final String title) {
    this.title = title;
  }
  @XmlElement
  public String getTitle() {
    return this.title;
  }
}

Then, you create a marshaller and ask it to convert an instance of class Book into XML:

final Book book = new Book("0132350882", "Clean Code");
final JAXBContext ctx = JAXBContext.newInstance(Book.class);
final Marshaller marshaller = ctx.createMarshaller();
marshaller.marshal(book, System.out);

Who is creating the XML? Not the book. Someone else, outside of the class Book. This is very wrong. Instead, this is how this should have been done. First, the class that has no idea about XML:

class DefaultBook implements Book {
  private final String title;
  DefaultBook(final String title) {
    this.title = title;
  }
  @Override
  public String getTitle() {
    return this.title;
  }
}

Then, the decorator that prints it to the XML:

class XmlBook implements Book{
  private final Book origin;
  XmlBook(final Book book) {
    this.origin = book;
  }
  @Override
  public String getTitle() {
    return this.origin.getTitle();
  }
  public String toXML() {
    return String.format(
      "<book><title>%s</title></book>",
      this.getTitle()
    );
  }
}

Now, in order to print the book in XML we do the following:

String xml = new XmlBook(
  new DefaultBook("Elegant Objects")
).toXML();

The XML printing functionality is inside XmlBook. If you don’t like the decorator idea, you can move the toXML() method to the DefaultBook class. It’s not important. What is important is that the functionality always stays where it belongs—inside the object. Only the object knows how to print itself to the XML. Nobody else!

@RetryOnFailure

Here is an example (from my own library):

import com.jcabi.aspects.RetryOnFailure;
class Foo {
  @RetryOnFailure
  public String load(URL url) {
    return url.openConnection().getContent();
  }
}

After compilation, we run a so called AOP weaver that technically turns our code into something like this:

class Foo {
  public String load(URL url) {
    while (true) {
      try {
        return _Foo.load(url);
      } catch (Exception ex) {
        // ignore it
      }
    }
  }
  class _Foo {
    public String load(URL url) {
      return url.openConnection().getContent();
    }
  }
}

I simplified the actual algorithm of retrying a method call on failure, but I’m sure you get the idea. AspectJ, the AOP engine, uses @RetryOnFailure annotation as a signal, informing us that the class has to be wrapped into another one. This is happening behind the scenes. We don’t see that supplementary class, which implements the retrying algorithm. But the bytecode produced by the AspectJ weaver contains a modified version of class Foo.

That is exactly what is wrong with this approach—we don’t see and don’t control the instantiation of that supplementary object. Object composition, which is the most important process in object design, is hidden somewhere behind the scenes. You may say that we don’t need to see it since it’s supplementary. I disagree. We must see how our objects are composed. We may not care about how they work, but we must see the entire composition process.

A much better design would look like this (instead of annotations):

Foo foo = new FooThatRetries(new Foo());

And then, the implementation of FooThatRetries:

class FooThatRetries implements Foo {
  private final Foo origin;
  FooThatRetries(Foo foo) {
    this.origin = foo;
  }
  public String load(URL url) {
    return new Retry().eval(
      new Retry.Algorithm<String>() {
        @Override
        public String eval() {
          return FooThatRetries.this.load(url);
        }
      }
    );
  }
}

And now, the implementation of Retry:

class Retry {
  public <T> T eval(Retry.Algorithm<T> algo) {
    while (true) {
      try {
        return algo.eval();
      } catch (Exception ex) {
        // ignore it
      }
    }
  }
  interface Algorithm<T> {
    T eval();
  }
}

Is the code longer? Yes. Is it cleaner? A lot more. I regret that I didn’t understand it two years ago, when I started to work with jcabi-aspects.


The bottom line is that annotations are bad. Don’t use them. What should be used instead? Object composition.

What could be worse than annotations? Configurations. For example, XML configurations. Spring XML configuration mechanisms is a perfect example of terrible design. I’ve said it many times before. Let me repeat it again—Spring Framework is one of the worst software products in the Java world. If you can stay away from it, you will do yourself a big favor.

There should not be any “configurations” in OOP. We can’t configure our objects if they are real objects. We can only instantiate them. And the best method of instantiation is operator new. This operator is the key instrument for an OOP developer. Taking it away from us and giving us “configuration mechanisms” is an unforgivable crime.

Annotations were introduced in Java 5, and we all got excited. Such a great instrument to make code shorter! No more Hibernate/Spring XML configuration files! Just annotations, right there in the code where we need them. No more marker interfaces, just a runtime-retained reflection-discoverable annotation! I was excited too. Moreover, I’ve made a few open source libraries which use annotations heavily. Take jcabi-aspects, for example. However, I’m not excited any more. Moreover, I believe that annotations are a big mistake in Java design.

Gomorra (2008) by Matteo Garrone)
Gomorra (2008) by Matteo Garrone)

Long story short, there is one big problem with annotations—they encourage us to implement object functionality outside of an object, which is against the very principle of encapsulation. The object is not solid any more, since its behavior is not defined entirely by its own methods—some of its functionality stays elsewhere. Why is it bad? Let’s see in a few examples.

@Inject

Say we annotate a property with @Inject:

import javax.inject.Inject;
public class Books {
  @Inject
  private final DB db;
  // some methods here, which use this.db
}

Then we have an injector that knows what to inject:

Injector injector = Guice.createInjector(
  new AbstractModule() {
    @Override
    public void configure() {
      this.bind(DB.class).toInstance(
        new Postgres("jdbc:postgresql:5740/main")
      );
    }
  }
);

Now we’re making an instance of class Books via the container:

Books books = injector.getInstance(Books.class);

The class Books has no idea how and who will inject an instance of class DB into it. This will happen behind the scenes and outside of its control. The injection will do it. It may look convenient, but this attitude causes a lot of damage to the entire code base. The control is lost (not inverted, but lost!). The object is not in charge any more. It can’t be responsible for what’s happening to it.

Instead, here is how this should be done:

class Books {
  private final DB db;
  Books(final DB base) {
    this.db = base;
  }
  // some methods here, which use this.db
}

This article explains why Dependency Injection containers are a wrong idea in the first place: Dependency Injection Containers are Code Polluters. Annotations basically provoke us to make the containers and use them. We move functionality outside of our objects and put it into containers, or somewhere else. That’s because we don’t want to duplicate the same code over and over again, right? That’s correct, duplication is bad, but tearing an object apart is even worse. Way worse. The same is true about ORM (JPA/Hibernate), where annotations are being actively used. Check this post, it explains what is wrong about ORM: ORM Is an Offensive Anti-Pattern. Annotations by themselves are not the key motivator, but they help us and encourage us by tearing objects apart and keeping parts in different places. They are containers, sessions, managers, controllers, etc.

@XmlElement

This is how JAXB works, when you want to convert your POJO to XML. First, you attach the @XmlElement annotation to the getter:

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
@XmlRootElement
public class Book {
  private final String title;
  public Book(final String title) {
    this.title = title;
  }
  @XmlElement
  public String getTitle() {
    return this.title;
  }
}

Then, you create a marshaller and ask it to convert an instance of class Book into XML:

final Book book = new Book("0132350882", "Clean Code");
final JAXBContext ctx = JAXBContext.newInstance(Book.class);
final Marshaller marshaller = ctx.createMarshaller();
marshaller.marshal(book, System.out);

Who is creating the XML? Not the book. Someone else, outside of the class Book. This is very wrong. Instead, this is how this should have been done. First, the class that has no idea about XML:

class DefaultBook implements Book {
  private final String title;
  DefaultBook(final String title) {
    this.title = title;
  }
  @Override
  public String getTitle() {
    return this.title;
  }
}

Then, the decorator that prints it to the XML:

class XmlBook implements Book{
  private final Book origin;
  XmlBook(final Book book) {
    this.origin = book;
  }
  @Override
  public String getTitle() {
    return this.origin.getTitle();
  }
  public String toXML() {
    return String.format(
      "<book><title>%s</title></book>",
      this.getTitle()
    );
  }
}

Now, in order to print the book in XML we do the following:

String xml = new XmlBook(
  new DefaultBook("Elegant Objects")
).toXML();

The XML printing functionality is inside XmlBook. If you don’t like the decorator idea, you can move the toXML() method to the DefaultBook class. It’s not important. What is important is that the functionality always stays where it belongs—inside the object. Only the object knows how to print itself to the XML. Nobody else!

@RetryOnFailure

Here is an example (from my own library):

import com.jcabi.aspects.RetryOnFailure;
class Foo {
  @RetryOnFailure
  public String load(URL url) {
    return url.openConnection().getContent();
  }
}

After compilation, we run a so called AOP weaver that technically turns our code into something like this:

class Foo {
  public String load(URL url) {
    while (true) {
      try {
        return _Foo.load(url);
      } catch (Exception ex) {
        // ignore it
      }
    }
  }
  class _Foo {
    public String load(URL url) {
      return url.openConnection().getContent();
    }
  }
}

I simplified the actual algorithm of retrying a method call on failure, but I’m sure you get the idea. AspectJ, the AOP engine, uses @RetryOnFailure annotation as a signal, informing us that the class has to be wrapped into another one. This is happening behind the scenes. We don’t see that supplementary class, which implements the retrying algorithm. But the bytecode produced by the AspectJ weaver contains a modified version of class Foo.

That is exactly what is wrong with this approach—we don’t see and don’t control the instantiation of that supplementary object. Object composition, which is the most important process in object design, is hidden somewhere behind the scenes. You may say that we don’t need to see it since it’s supplementary. I disagree. We must see how our objects are composed. We may not care about how they work, but we must see the entire composition process.

A much better design would look like this (instead of annotations):

Foo foo = new FooThatRetries(new Foo());

And then, the implementation of FooThatRetries:

class FooThatRetries implements Foo {
  private final Foo origin;
  FooThatRetries(Foo foo) {
    this.origin = foo;
  }
  public String load(URL url) {
    return new Retry().eval(
      new Retry.Algorithm<String>() {
        @Override
        public String eval() {
          return FooThatRetries.this.load(url);
        }
      }
    );
  }
}

And now, the implementation of Retry:

class Retry {
  public <T> T eval(Retry.Algorithm<T> algo) {
    while (true) {
      try {
        return algo.eval();
      } catch (Exception ex) {
        // ignore it
      }
    }
  }
  interface Algorithm<T> {
    T eval();
  }
}

Is the code longer? Yes. Is it cleaner? A lot more. I regret that I didn’t understand it two years ago, when I started to work with jcabi-aspects.


The bottom line is that annotations are bad. Don’t use them. What should be used instead? Object composition.

What could be worse than annotations? Configurations. For example, XML configurations. Spring XML configuration mechanisms is a perfect example of terrible design. I’ve said it many times before. Let me repeat it again—Spring Framework is one of the worst software products in the Java world. If you can stay away from it, you will do yourself a big favor.

There should not be any “configurations” in OOP. We can’t configure our objects if they are real objects. We can only instantiate them. And the best method of instantiation is operator new. This operator is the key instrument for an OOP developer. Taking it away from us and giving us “configuration mechanisms” is an unforgivable crime.

© Yegor Bugayenko 2014–2018

Printers Instead of Getters

QR code

Printers Instead of Getters

  • Palo Alto, CA
  • comments

Getters and setters are evil. No need to argue about this, it’s settled. You disagree? Let’s discuss that later. For now, let’s say, we want to get rid of getters. The key question is how is it possible at all? We do need to get the data out of an object, right? Nope. Wrong.

Le fabuleux destin d'Amélie Poulain (2001) by Jean-Pierre Jeunet
Le fabuleux destin d'Amélie Poulain (2001) by Jean-Pierre Jeunet

I’m suggesting to use “printers” instead. Instead of exposing data via getters, an object will have a functionality of printing itself to some media.

Let’s say this is our class:

public class Book {
  private final String isbn =
    "0735619654";
  private final String title =
    "Object Thinking";
}

We need it to be transferred into XML format. A more or less traditional way to do it is via getters and JAXB:

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
@XmlRootElement
public class Book {
  private final String isbn =
    "0735619654";
  private final String title =
    "Object Thinking";
  @XmlElement
  public String getIsbn() {
    return this.isbn;
  }
  @XmlElement
  public String getTitle() {
    return this.title;
  }
}

This is a very offensive way of treating the object. We’re basically exposing everything that’s inside to the public. It was a nice little self-sufficient solid object and we turned it into a bag of data, which anyone can access in many possible ways. We can access it for reading, of course.

It is convenient to have these getters, you may say. We are all used to them. If we want to convert it into JSON, they will be very helpful. If we want to use this poor object as a data object in JSP, getters will help us. There are many examples in Java, where getters are being actively used.

This is not because they are so effective. This is because we’re so procedural in our way of thinking. We don’t trust our objects. We only trust the data they store. We don’t want this Book object to generate the XML. We want it to give us the data. We will build the XML. The Book is too stupid to do that job. We’re way smarter!

I’m suggesting to stop thinking this way. Instead, let’s try to give this poor Book a chance, and equip it with a “printer”:

public class Book {
  private final String isbn =
    "0735619654";
  private final String title =
    "Object Thinking";
  public String toXML() {
    return String.format(
      "<book><isbn>%s</isbn><title>%s</title></book>",
      this.isbn, this.title
    );
  }
}

This isn’t the best implementation, but you got the idea. The object is not exposing its internals any more. We can’t get its ISBN and its title. We can only ask it to print itself in XML format.

We can add an additional printer, if another format is required:

public class Book {
  private final String isbn =
    "0735619654";
  private final String title =
    "Object Thinking";
  public String toJSON() {
    return String.format(
      "{\"isbn\":\"%s\", \"title\":\"%s\"}",
      this.isbn, this.title
    );
  }
}

Again, not the best implementation, but you see what I’m trying to show. Each time we need a new format, we create a new printer.

You may say that the object will be rather big if there will be many formats. That’s true, but a big object is a bad design in the first place. I would say that if there is more than one printer—it’s a problem.

So, what to do if we need multiple formats? Use “media,” where that printers will be able to print to. Say, we have an object that represents a record in MySQL. We want it to be printable to XML, HTML, JSON, some binary format and God knows what else. We can add that many printers to it, but the object will be big and ugly. To avoid that, introduce a new object, that represents the media where the data will be printed to:

public class Book {
  private final String isbn =
    "0735619654";
  private final String title =
    "Object Thinking";
  public Media print(Media media) {
    return media
      .with("isbn", this.isbn)
      .with("title", this.title);
  }
}

Again, it’s a very primitive design of that immutable Media class, but you got the idea—the media accepts the data. Now, we want to print our object to JSON (this design is not really perfect, since JsonObjectBuilder is not immutable, even though it looks like one…):

class JsonMedia implements Media {
  private final JsonObjectBuilder builder;
  JsonMedia() {
    this("book");
  }
  JsonMedia(String head) {
    this(Json.createObjectBuilder().add(head));
  }
  JsonMedia(JsonObjectBuilder bdr) {
    this.builder = bdr;
  }
  @Override
  public Media with(String name, String value) {
    return new JsonMedia(
      this.builder.add(name, value)
    );
  }
  public JsonObject json() {
    return this.builder.build();
  }
}

Now, we make an instance of JsonMedia and ask our book to print itself there:

JsonMedia media = new JsonMedia("book");
book.print(media);
JsonObject json = media.json();

Voilà! The JSON object is ready and the book has no idea about what exactly what printed just now. We need to print the book to XML? We create XmlMedia, which will print the book to XML. The Book class stays small, while the complexity of “media” objects is unlimited.

My point here is simple—no getters, just printers!

Getters and setters are evil. No need to argue about this, it’s settled. You disagree? Let’s discuss that later. For now, let’s say, we want to get rid of getters. The key question is how is it possible at all? We do need to get the data out of an object, right? Nope. Wrong.

Le fabuleux destin d'Amélie Poulain (2001) by Jean-Pierre Jeunet
Le fabuleux destin d'Amélie Poulain (2001) by Jean-Pierre Jeunet

I’m suggesting to use “printers” instead. Instead of exposing data via getters, an object will have a functionality of printing itself to some media.

Let’s say this is our class:

public class Book {
  private final String isbn =
    "0735619654";
  private final String title =
    "Object Thinking";
}

We need it to be transferred into XML format. A more or less traditional way to do it is via getters and JAXB:

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
@XmlRootElement
public class Book {
  private final String isbn =
    "0735619654";
  private final String title =
    "Object Thinking";
  @XmlElement
  public String getIsbn() {
    return this.isbn;
  }
  @XmlElement
  public String getTitle() {
    return this.title;
  }
}

This is a very offensive way of treating the object. We’re basically exposing everything that’s inside to the public. It was a nice little self-sufficient solid object and we turned it into a bag of data, which anyone can access in many possible ways. We can access it for reading, of course.

It is convenient to have these getters, you may say. We are all used to them. If we want to convert it into JSON, they will be very helpful. If we want to use this poor object as a data object in JSP, getters will help us. There are many examples in Java, where getters are being actively used.

This is not because they are so effective. This is because we’re so procedural in our way of thinking. We don’t trust our objects. We only trust the data they store. We don’t want this Book object to generate the XML. We want it to give us the data. We will build the XML. The Book is too stupid to do that job. We’re way smarter!

I’m suggesting to stop thinking this way. Instead, let’s try to give this poor Book a chance, and equip it with a “printer”:

public class Book {
  private final String isbn =
    "0735619654";
  private final String title =
    "Object Thinking";
  public String toXML() {
    return String.format(
      "<book><isbn>%s</isbn><title>%s</title></book>",
      this.isbn, this.title
    );
  }
}

This isn’t the best implementation, but you got the idea. The object is not exposing its internals any more. We can’t get its ISBN and its title. We can only ask it to print itself in XML format.

We can add an additional printer, if another format is required:

public class Book {
  private final String isbn =
    "0735619654";
  private final String title =
    "Object Thinking";
  public String toJSON() {
    return String.format(
      "{\"isbn\":\"%s\", \"title\":\"%s\"}",
      this.isbn, this.title
    );
  }
}

Again, not the best implementation, but you see what I’m trying to show. Each time we need a new format, we create a new printer.

You may say that the object will be rather big if there will be many formats. That’s true, but a big object is a bad design in the first place. I would say that if there is more than one printer—it’s a problem.

So, what to do if we need multiple formats? Use “media,” where that printers will be able to print to. Say, we have an object that represents a record in MySQL. We want it to be printable to XML, HTML, JSON, some binary format and God knows what else. We can add that many printers to it, but the object will be big and ugly. To avoid that, introduce a new object, that represents the media where the data will be printed to:

public class Book {
  private final String isbn =
    "0735619654";
  private final String title =
    "Object Thinking";
  public Media print(Media media) {
    return media
      .with("isbn", this.isbn)
      .with("title", this.title);
  }
}

Again, it’s a very primitive design of that immutable Media class, but you got the idea—the media accepts the data. Now, we want to print our object to JSON (this design is not really perfect, since JsonObjectBuilder is not immutable, even though it looks like one…):

class JsonMedia implements Media {
  private final JsonObjectBuilder builder;
  JsonMedia() {
    this("book");
  }
  JsonMedia(String head) {
    this(Json.createObjectBuilder().add(head));
  }
  JsonMedia(JsonObjectBuilder bdr) {
    this.builder = bdr;
  }
  @Override
  public Media with(String name, String value) {
    return new JsonMedia(
      this.builder.add(name, value)
    );
  }
  public JsonObject json() {
    return this.builder.build();
  }
}

Now, we make an instance of JsonMedia and ask our book to print itself there:

JsonMedia media = new JsonMedia("book");
book.print(media);
JsonObject json = media.json();

Voilà! The JSON object is ready and the book has no idea about what exactly what printed just now. We need to print the book to XML? We create XmlMedia, which will print the book to XML. The Book class stays small, while the complexity of “media” objects is unlimited.

My point here is simple—no getters, just printers!

© Yegor Bugayenko 2014–2018

Try. Finally. If. Not. Null.

QR code

Try. Finally. If. Not. Null.

  • Palo Alto, CA
  • comments

There is a very typical mistake in pre-Java7 “try/finally” scenario, which I keep seeing in so many code reviews. I just have to write about it. Java7 introduced a solution, but it doesn’t cover all situations. Sometimes we need to deal with non-AutoCloseable resources. Let’s open and close them correctly, please.

Lock, Stock and Two Smoking Barrels (1998) by Guy Ritchie
Lock, Stock and Two Smoking Barrels (1998) by Guy Ritchie

This is how it looks (assuming we are in Java 6):

InputStream input = null;
try {
  input = url.openStream();
  // reads the stream, throws IOException
} catch (IOException ex) {
  throw new RuntimeException(ex);
} finally {
  if (input != null) {
    input.close();
  }
}

I already wrote about null and its evil nature. Here it comes again. If you just follow the rule of “not using NULL anywhere ever,” this code would need an immediate refactoring. Its correct version will look like this:

final InputStream input = url.openStream();
try {
  // reads the stream, throws IOException
} catch (IOException ex) {
  throw new RuntimeException(ex);
} finally {
  input.close();
}

There is no null anymore and it’s very clean. Isn’t it?

There are situations when opening the resource itself throws IOException and we can’t put it outside of try/catch. In that case, we have to have two try/catch blocks:

final InputStream input;
try {
  input = url.openStream();
} catch (IOException ex) {
  throw new RuntimeException(ex);
}
try {
  // reads the stream, throws IOException
} catch (IOException ex) {
  throw new RuntimeException(ex);
} finally {
  input.close();
}

But there should be no null, never!

The presence of null in Java code is a clear indicator of code smell. Something is not right if you have to use null. The only place where the presence of null is justified is where we’re using third-party APIs or JDK. They may return null sometimes because… well, their design is bad. We have no other option but to do if(x==null). But that’s it. No other places are good for null.

There is a very typical mistake in pre-Java7 “try/finally” scenario, which I keep seeing in so many code reviews. I just have to write about it. Java7 introduced a solution, but it doesn’t cover all situations. Sometimes we need to deal with non-AutoCloseable resources. Let’s open and close them correctly, please.

Lock, Stock and Two Smoking Barrels (1998) by Guy Ritchie
Lock, Stock and Two Smoking Barrels (1998) by Guy Ritchie

This is how it looks (assuming we are in Java 6):

InputStream input = null;
try {
  input = url.openStream();
  // reads the stream, throws IOException
} catch (IOException ex) {
  throw new RuntimeException(ex);
} finally {
  if (input != null) {
    input.close();
  }
}

I already wrote about null and its evil nature. Here it comes again. If you just follow the rule of “not using NULL anywhere ever,” this code would need an immediate refactoring. Its correct version will look like this:

final InputStream input = url.openStream();
try {
  // reads the stream, throws IOException
} catch (IOException ex) {
  throw new RuntimeException(ex);
} finally {
  input.close();
}

There is no null anymore and it’s very clean. Isn’t it?

There are situations when opening the resource itself throws IOException and we can’t put it outside of try/catch. In that case, we have to have two try/catch blocks:

final InputStream input;
try {
  input = url.openStream();
} catch (IOException ex) {
  throw new RuntimeException(ex);
}
try {
  // reads the stream, throws IOException
} catch (IOException ex) {
  throw new RuntimeException(ex);
} finally {
  input.close();
}

But there should be no null, never!

The presence of null in Java code is a clear indicator of code smell. Something is not right if you have to use null. The only place where the presence of null is justified is where we’re using third-party APIs or JDK. They may return null sometimes because… well, their design is bad. We have no other option but to do if(x==null). But that’s it. No other places are good for null.

© Yegor Bugayenko 2014–2018

Temporal Coupling Between Method Calls

QR code

Temporal Coupling Between Method Calls

  • Kiev, Ukraine
  • comments

Temporal coupling happens between sequential method calls when they must stay in a particular order. This is inevitable in imperative programming, but we can reduce the negative effect of it just by turning those static procedures into functions. Take a look at this example.

Blueberry (2004) by Jan Kounen
Blueberry (2004) by Jan Kounen

Here is the code:

class Foo {
  public List<String> names() {
    List<String> list = new LinkedList();
    Foo.append(list, "Jeff");
    Foo.append(list, "Walter");
    return list;
  }
  private static void append(
    List<String> list, String item) {
    list.add(item.toLowerCase());
  }
}

What do you think about that? I believe it’s clear what names() is doing—creating a list of names. In order to avoid duplication, there is a supplementary procedure, append(), which converts an item to lowercase and adds it to the list.

This is poor design.

It is a procedural design, and there is temporal coupling between lines in method names().

Let me first show you a better (though not the best!) design, then I will try to explain its benefits:

class Foo {
  public List<String> names() {
    return Foo.with(
      Foo.with(
        new LinkedList(),
        "Jeff"
      ),
      "Walter"
    );
  }
  private static List<String> with(
    List<String> list, String item) {
    list.add(item.toLowerCase());
    return list;
  }
}

An ideal design for method with() would create a new instance of List, populate it through addAll(list), then add(item) to it, and finally return. That would be perfectly immutable, but slow.

So, what is wrong with this:

List<String> list = new LinkedList();
Foo.append(list, "Jeff");
Foo.append(list, "Walter");
return list;

It looks perfectly clean, doesn’t it? Instantiate a list, append two items to it, and return it. Yes, it is clean—for now. Because we remember what append() is doing. In a few months, we’ll get back to this code, and it will look like this:

List<String> list = new LinkedList();
// 10 more lines here
Foo.append(list, "Jeff");
Foo.append(list, "Walter");
// 10 more lines here
return list;

Is it so clear now that append() is actually adding "Jeff" to list? What will happen if I remove that line? Will it affect the result being returned in the last line? I don’t know. I need to check the body of method append() to make sure.

Also, how about returning list first and calling append() afterwards? This is what possible “refactoring” may do to our code:

List<String> list = new LinkedList();
if (/* something */) {
  return list;
}
// 10 more lines here
Foo.append(list, "Walter");
Foo.append(list, "Jeff");
// 10 more lines here
return list;

First of all, we return list too early, when it is not ready. But did anyone tell me that these two calls to append() must happen before return list? Second, we changed the order of append() calls. Again, did anyone tell me that it’s important to call them in that particular order?

Nobody. Nowhere. This is called temporal coupling.

Our lines are coupled together. They must stay in this particular order, but the knowledge about that order is hidden. It’s easy to destroy the order, and our compiler won’t be able to catch us.

To the contrary, this design doesn’t have any “order”:

return Foo.with(
  Foo.with(
    new LinkedList(),
    "Jeff"
  ),
  "Walter"
);

It just returns a list, which is constructed by a few calls to the with() method. It is a single line instead of four.

As discussed before, an ideal method in OOP must have just a single statement, and this statement is return.

The same is true about validation. For example, this code is bad:

list.add("Jeff");
Foo.checkIfListStillHasSpace(list);
list.add("Walter");

While this one is much better:

list.add("Jeff");
Foo.withEnoughSpace(list).add("Walter");

See the difference?

And, of course, an ideal approach would be to use composable decorators instead of these ugly static methods. But if it’s not possible for some reason, just don’t make those static methods look like procedures. Make sure they always return results, which become arguments to further calls.

Temporal coupling happens between sequential method calls when they must stay in a particular order. This is inevitable in imperative programming, but we can reduce the negative effect of it just by turning those static procedures into functions. Take a look at this example.

Blueberry (2004) by Jan Kounen
Blueberry (2004) by Jan Kounen

Here is the code:

class Foo {
  public List<String> names() {
    List<String> list = new LinkedList();
    Foo.append(list, "Jeff");
    Foo.append(list, "Walter");
    return list;
  }
  private static void append(
    List<String> list, String item) {
    list.add(item.toLowerCase());
  }
}

What do you think about that? I believe it’s clear what names() is doing—creating a list of names. In order to avoid duplication, there is a supplementary procedure, append(), which converts an item to lowercase and adds it to the list.

This is poor design.

It is a procedural design, and there is temporal coupling between lines in method names().

Let me first show you a better (though not the best!) design, then I will try to explain its benefits:

class Foo {
  public List<String> names() {
    return Foo.with(
      Foo.with(
        new LinkedList(),
        "Jeff"
      ),
      "Walter"
    );
  }
  private static List<String> with(
    List<String> list, String item) {
    list.add(item.toLowerCase());
    return list;
  }
}

An ideal design for method with() would create a new instance of List, populate it through addAll(list), then add(item) to it, and finally return. That would be perfectly immutable, but slow.

So, what is wrong with this:

List<String> list = new LinkedList();
Foo.append(list, "Jeff");
Foo.append(list, "Walter");
return list;

It looks perfectly clean, doesn’t it? Instantiate a list, append two items to it, and return it. Yes, it is clean—for now. Because we remember what append() is doing. In a few months, we’ll get back to this code, and it will look like this:

List<String> list = new LinkedList();
// 10 more lines here
Foo.append(list, "Jeff");
Foo.append(list, "Walter");
// 10 more lines here
return list;

Is it so clear now that append() is actually adding "Jeff" to list? What will happen if I remove that line? Will it affect the result being returned in the last line? I don’t know. I need to check the body of method append() to make sure.

Also, how about returning list first and calling append() afterwards? This is what possible “refactoring” may do to our code:

List<String> list = new LinkedList();
if (/* something */) {
  return list;
}
// 10 more lines here
Foo.append(list, "Walter");
Foo.append(list, "Jeff");
// 10 more lines here
return list;

First of all, we return list too early, when it is not ready. But did anyone tell me that these two calls to append() must happen before return list? Second, we changed the order of append() calls. Again, did anyone tell me that it’s important to call them in that particular order?

Nobody. Nowhere. This is called temporal coupling.

Our lines are coupled together. They must stay in this particular order, but the knowledge about that order is hidden. It’s easy to destroy the order, and our compiler won’t be able to catch us.

To the contrary, this design doesn’t have any “order”:

return Foo.with(
  Foo.with(
    new LinkedList(),
    "Jeff"
  ),
  "Walter"
);

It just returns a list, which is constructed by a few calls to the with() method. It is a single line instead of four.

As discussed before, an ideal method in OOP must have just a single statement, and this statement is return.

The same is true about validation. For example, this code is bad:

list.add("Jeff");
Foo.checkIfListStillHasSpace(list);
list.add("Walter");

While this one is much better:

list.add("Jeff");
Foo.withEnoughSpace(list).add("Walter");

See the difference?

And, of course, an ideal approach would be to use composable decorators instead of these ugly static methods. But if it’s not possible for some reason, just don’t make those static methods look like procedures. Make sure they always return results, which become arguments to further calls.

© Yegor Bugayenko 2014–2018

Throwing an Exception Without Proper Context Is a Bad Habit

QR code

Throwing an Exception Without Proper Context Is a Bad Habit

  • San Jose, CA
  • comments

I keep repeating the same mistake again and again. So it’s time to stop and make a rule to prevent this from happening anymore. The mistake is not fatal, but it’s very annoying. When I look at production logs, I often see something like "File doesn't exist", and I ask myself: What file? Where is it supposed to exist? What did the server try to do with it? What was going on a second before it crashed? There is no answer in the log, and it’s totally my fault. I either 1) don’t re-throw or 2) re-throw without providing context. Both are wrong.

Four Rooms (1995) by Allison Anders et al.
Four Rooms (1995) by Allison Anders et al.

This is how the code may look:

if (!file.exists()) {
  throw new IllegalArgumentException(
    "File doesn't exist"
  );
}

It may also look like this:

try {
  Files.delete(file);
} catch (IOException ex) {
  throw new IllegalArgumentException(ex);
}

Both examples demonstrate an inadequate style of handling situations that involve exceptions and reporting them. What’s wrong here? The exception messages are not thorough enough. They simply don’t contain any information from the place where they originated from.

This is how they should look instead:

if (!file.exists()) {
  throw new IllegalArgumentException(
    String.format(
      "User profile file %s doesn't exist",
      file.getAbsolutePath()
    )
  );
}

And the second example should look like this:

try {
  Files.delete(file);
} catch (IOException ex) {
  throw new IllegalArgumentException(
    String.format(
      "Can't delete user profile data file %s",
      file.getAbsolutePath()
    ),
    ex
  );
}

See the difference? This may look like redundant code, but it’s not. Of course, when I’m writing all this, I don’t really care about logs and exceptions. I’m not really expecting this file to be absent.

But I should.

There should be a rule: Every time we throw or re-throw, an exception message must describe the problem with as much detail as possible.

Of course, we can’t forget about security and risk putting any sensitive information into the exception message, like passwords, credit card numbers, etc. Besides that, as much as possible must be exposed to the exception catcher at a higher level.

Throwing an exception is literally an escalation of a problem to a higher level of management. Imagine that my boss is asking me to install a new server. I come back to him in a few hours and say, “I failed; sorry.” That would sound strange. He would ask for more details. Why did I fail? What exactly went wrong? Is it possible to do it differently? Etc.

Such code is literally a sign of disrespect to the client:

throw new IllegalArgumentException(
  "File doesn't exist"
);

I have to be more verbose and give more details.

And I’m not alone in this mistake. I see it everywhere, and it really makes debugging difficult, especially in production, where it’s almost impossible to reproduce the problem right away.

Thus, please be more verbose in your exception messages. I will do the same in my code :)

And one more thing before you go. In most OOP languages, exceptions are unchecked, which means that catching them is not a mandatory operation, unfortunately. Nevertheless, I recommend you catch, add context, and re-throw them all, always. This may seem like pure noise, but it’s not! Just make your methods smaller and ensure all exceptions sent out of them have enough information about their origins. You will do yourself and everybody else a big favor.

I keep repeating the same mistake again and again. So it’s time to stop and make a rule to prevent this from happening anymore. The mistake is not fatal, but it’s very annoying. When I look at production logs, I often see something like "File doesn't exist", and I ask myself: What file? Where is it supposed to exist? What did the server try to do with it? What was going on a second before it crashed? There is no answer in the log, and it’s totally my fault. I either 1) don’t re-throw or 2) re-throw without providing context. Both are wrong.

Four Rooms (1995) by Allison Anders et al.
Four Rooms (1995) by Allison Anders et al.

This is how the code may look:

if (!file.exists()) {
  throw new IllegalArgumentException(
    "File doesn't exist"
  );
}

It may also look like this:

try {
  Files.delete(file);
} catch (IOException ex) {
  throw new IllegalArgumentException(ex);
}

Both examples demonstrate an inadequate style of handling situations that involve exceptions and reporting them. What’s wrong here? The exception messages are not thorough enough. They simply don’t contain any information from the place where they originated from.

This is how they should look instead:

if (!file.exists()) {
  throw new IllegalArgumentException(
    String.format(
      "User profile file %s doesn't exist",
      file.getAbsolutePath()
    )
  );
}

And the second example should look like this:

try {
  Files.delete(file);
} catch (IOException ex) {
  throw new IllegalArgumentException(
    String.format(
      "Can't delete user profile data file %s",
      file.getAbsolutePath()
    ),
    ex
  );
}

See the difference? This may look like redundant code, but it’s not. Of course, when I’m writing all this, I don’t really care about logs and exceptions. I’m not really expecting this file to be absent.

But I should.

There should be a rule: Every time we throw or re-throw, an exception message must describe the problem with as much detail as possible.

Of course, we can’t forget about security and risk putting any sensitive information into the exception message, like passwords, credit card numbers, etc. Besides that, as much as possible must be exposed to the exception catcher at a higher level.

Throwing an exception is literally an escalation of a problem to a higher level of management. Imagine that my boss is asking me to install a new server. I come back to him in a few hours and say, “I failed; sorry.” That would sound strange. He would ask for more details. Why did I fail? What exactly went wrong? Is it possible to do it differently? Etc.

Such code is literally a sign of disrespect to the client:

throw new IllegalArgumentException(
  "File doesn't exist"
);

I have to be more verbose and give more details.

And I’m not alone in this mistake. I see it everywhere, and it really makes debugging difficult, especially in production, where it’s almost impossible to reproduce the problem right away.

Thus, please be more verbose in your exception messages. I will do the same in my code :)

And one more thing before you go. In most OOP languages, exceptions are unchecked, which means that catching them is not a mandatory operation, unfortunately. Nevertheless, I recommend you catch, add context, and re-throw them all, always. This may seem like pure noise, but it’s not! Just make your methods smaller and ensure all exceptions sent out of them have enough information about their origins. You will do yourself and everybody else a big favor.

© Yegor Bugayenko 2014–2018

A Chatbot Is Better Than a UI for a Microservice

QR code

A Chatbot Is Better Than a UI for a Microservice

  • Seattle, WA
  • comments

A chatbot (or chatterbot, as Wikipedia says) is a piece of software that talks to you in chat format. We use chatbots in a few (micro)services, and they fully replace user interfaces. I don’t think there is any innovation in this approach, but it has proved to be very effective over the last year or so. That’s the impetus for this post. Here is how the Rultor chatbot works for us and what its benefits are.

Let me give an example first. Look at the jcabi/jcabi-http#115 GitHub ticket:

The figure

Let’s see what’s going on here, and then we’ll discuss how it’s designed inside. Essentially, I’m talking to a chatbot here. The name of the chatbot is @rultor (I wrote about it last year). At 1, I’m asking the chatbot to release a new version of the jcabi-http library. At 2, the chatbot responds, just confirming that the task is clear and that it’s on it. At 3, the bot says the job is completed and its completion took nine minutes. Our conversation is over. That’s it.

Now, what is so special about this?

One thing: There is no user interface. Well, there is no traditional web-based HTML/CSS user interface. There is no login, logout, profile, menu, or anything like this. Rultor is a web service that has no web UI. The only way to communicate with it is by talking with its chatbot.

What’s so good about it? A few things.

Service Is Not a Server

This is how the traditional architecture of a web system would look:

PlantUML SVG diagram

A user gives instructions to a service and receives responses. This communication happens through a user interface (UI)—a bunch of HTTP entry points that receive requests from a browser and return HTML+CSS responses. Or, if a user is on another service, requests may contain some data, and responses will be in XML or JSON. You get the idea; a user is a client, and the service is a server.

Like in a restaurant—you say what you want, and a server goes to the kitchen, waits there, and in a few minutes, comes back with spaghetti carbonara. You’re a client, and that cute lady is a server.

In the case of a chatbot, that’s not the case anymore. Look at the architecture:

PlantUML SVG diagram

First, a user posts a request to GitHub through a web user interface provided by GitHub. It is a communication hub for us. Then, the service connects to GitHub through its RESTful API and checks whether there are any new requests there. If something new is found, the service does the job, prepares a response, and posts it there. The client receives an email notification about a new response just posted to the ticket. The client then checks GitHub and finds the response.

badge

Here is how this would look in a restaurant: There would be a board with sticky notes. First, you write the note, “I’d like spaghetti carbonara with parmesan and fresh pepper on top” (Damn, I’m just too hungry now), and pin it to the board at number 15. Then, you return to your table. A chef from the kitchen checks that board and finds your sticky note. He makes that spaghetti, tops it with parmesan, fresh pepper, some basil leaves, and virgin olive oil … yeah, he makes it right … and puts it next to the board. You hear an announcement that order number 15 is ready. You go there, collect the food, return to your table, and enjoy.

The point is that there is no cute lady involved anymore. There is no server. There are two parties communicating with the board—you and the kitchen. The kitchen is our microservice, but it’s not a server anymore.

These two parties are perfectly decoupled now. They never talk to each other. And they both are clients of the communication hub, which is GitHub or a board in the restaurant.

Again, the microservice is not a server anymore. Instead, it is a client of a communication hub. And the flip of its position provides a lot of benefits to us, its developers.

No Need to Be Fast

badge

First of all, we don’t need to care much about the performance of our UI. Well, we don’t care at all, since we don’t have a UI. Do we care about the speed of responses on GitHub? Not really. When a user posts a message to GitHub, he or she doesn’t expect our chatbot to give an immediate answer in less than 100 milliseconds. (That’s what any properly designed web system must guarantee, I believe.)

We put a note on the board, and we assume that the kitchen is probably doing something else at the moment. We’ll wait for a few seconds or even minutes. If, on the other hand, I give an order to the waitress and she waits five seconds before replying back, I’ll be very surprised. If she keeps doing that with every question, I’ll start to wonder to myself if everything is OK with her.

I expect a user interface to be instant, while in a chat I have no problem allowing some time for the bot to answer. This happens naturally. We’re used to delays in chats, when we’re talking with real people. They need some time to process our information, to think, and to type something back.

But a user interface doesn’t have that luxury. It has to be bullet-fast; otherwise, I immediately get frustrated. The same thing happens to you, right?

No Need to Look Cute

badge

Another advantage of this no-server design is that there is no need to look pretty. There is no web interface, no HTML, no CSS, no graphic design. Perhaps not everybody really likes that. Most non-professional users may still prefer to talk to a cute server instead of sticking some paper notes to the board. But if we’re dealing with professional computer engineers, they’re not that demanding.

Rultor doesn’t have any web UI, and its users simply don’t know how it “looks.” It just talks to you. The only thing you see is its avatar in GitHub.

This saves a lot of money and time on design efforts, which are usually very expensive if you aim for high quality. If your web service looks average, most of its users will assume that it also works average. Many good ideas have simply died because their UI wasn’t as impressive as people were used to, thanks to all those Pinterests and Instagrams.

A good-looking server has a greater chance for bigger tips, right? If there is no server and we don’t see the chef, we judge him or her only by the quality of the food.

Same here. By getting rid of a UI, we allow ourselves to focus on the quality of the service we’re delivering. We don’t burn our time and money on being nice. We spend them on being useful.

Much Easier to Scale

badge

If we have too many stickies on that board, we just hire more cooks, or maybe even build another kitchen, and the problem is solved. We can handle as many customers as necessary. Well, as long as the board is powerful enough to handle multiple parallel users.

GitHub is a pretty big platform, with hundreds of thousands of users and projects. If we have too many requests coming in, we can just add more processing nodes to Rultor. Remember, we’re not a server anymore; we are a client of GitHub. We decide when to connect to GitHub and when to create responses to the requests submitted.

It is much easier to create a scalable client than a scalable server, mostly because there is nobody really waiting for us to respond quickly. The load of requests we’re getting can be managed much easier, since the decision of when to process them is made by us.

Mistakes Are Not So Visible

badge

When you’re standing in front of a customer, most of your mistakes are unforgivable, primarily because they are very visible. On the other hand, when you’re cooking something in the kitchen, nobody can see you and spot your faults. They will only spot them if the spaghetti has too much salt. In other words, they will judge you by your results, not by how you produce them.

It’s the same story with the microservice. When it works as a server, we expect it to be seamless, respond immediately, and present everything in a structured and organized way. If something goes wrong, it’s right here on the web page. Your best case is a 404, while the worst one is that you present some wrong information to the user. Even though the bug may not be critical inside the microservice engine, the user doesn’t know that. The user will judge you by your appearance and won’t forget even small mistakes.

However, when you both are clients of a message board, you don’t see each other. The user communicates with GitHub, and the microservice interacts with GitHub. Mistakes are less visible. Trust me, we have had many of them over the 18 months that Rultor has been in public use. We’ve had downtimes, we’ve had serious logical mistakes, and we’ve had data corruption. But very rarely have these problems become visible online. We merely saw them in our server logs. Users didn’t see them. Well, mostly :)

Everything Is Traceable

badge

Since there is a communication board between us, it’s very easy to see the entire history of our discussion, which is very intuitive. It’s like a Slack chat history. You see what we started from, who said what, and which conclusions were made.

Basically, you can’t have that visibility in a web UI. Well, you can probably create a special page with the “history of operations,” but who would check it? And how visible and simple would that information be? And, what’s most important, how would that information match up with the UI?

In the log, you’ll state that “the build was started,” but what’s the build and how was it started? How can I start it again? Using which buttons and web controls? It’s not clear.

Thus, the traceability of a chronological chat is unbeatable.

Easy to Integrate With Other Services

Yes, think about the future of this approach. If there is a centralized message board where users talk to a chatbot, why can’t other chatbots talk to each other too?

Forget about RESTful APIs. Just a message board where chatbots post their requests and collect responses. They are perfectly decoupled, replaceable, and very scalable. Also, their communication protocol is visible and very traceable. And they boast many other benefits, as was just explained above. It’s much more convenient for us, both users and programmers, to monitor them and create them.

Well, maybe it’s too extreme to get rid of RESTful APIs entirely, but to some extent, this approach is feasible, I believe.

I didn’t go too far with this idea, but something was done. We have a messaging platform that allows multiple chatbots to communicate with users. It’s called Netbout. It’s a very primitive web system with isolated discussions. Simply put, anyone can create a new discussion, invite a few friends, and post messages there. Both users and chatbots can do that.

So, when a new candidate wants to join Zerocracy, we ask that person to fill out an online form. When the candidate clicks the “Submit” button, a new discussion starts, and the first chatbot decides who should interview that person. The decision is made according to the skills listed in the form. The chatbot invites one of our best programmers to conduct the interview. When the interview is done, another chatbot explains to the candidate what the next steps are, registers him or her in our database, and starts to show the progress of work.

From a user perspective, it looks like he or she is talking to a few people who understand just a few simple commands. It is very intuitive and was easy to design.

I think chatbots are a good approach for interacting with microservices. Especially when users are more or less professional.

PS. Illustrations by Kristina Wheat.

A chatbot (or chatterbot, as Wikipedia says) is a piece of software that talks to you in chat format. We use chatbots in a few (micro)services, and they fully replace user interfaces. I don’t think there is any innovation in this approach, but it has proved to be very effective over the last year or so. That’s the impetus for this post. Here is how the Rultor chatbot works for us and what its benefits are.

Let me give an example first. Look at the jcabi/jcabi-http#115 GitHub ticket:

The figure

Let’s see what’s going on here, and then we’ll discuss how it’s designed inside. Essentially, I’m talking to a chatbot here. The name of the chatbot is @rultor (I wrote about it last year). At 1, I’m asking the chatbot to release a new version of the jcabi-http library. At 2, the chatbot responds, just confirming that the task is clear and that it’s on it. At 3, the bot says the job is completed and its completion took nine minutes. Our conversation is over. That’s it.

Now, what is so special about this?

One thing: There is no user interface. Well, there is no traditional web-based HTML/CSS user interface. There is no login, logout, profile, menu, or anything like this. Rultor is a web service that has no web UI. The only way to communicate with it is by talking with its chatbot.

What’s so good about it? A few things.

Service Is Not a Server

This is how the traditional architecture of a web system would look:

PlantUML SVG diagram

A user gives instructions to a service and receives responses. This communication happens through a user interface (UI)—a bunch of HTTP entry points that receive requests from a browser and return HTML+CSS responses. Or, if a user is on another service, requests may contain some data, and responses will be in XML or JSON. You get the idea; a user is a client, and the service is a server.

Like in a restaurant—you say what you want, and a server goes to the kitchen, waits there, and in a few minutes, comes back with spaghetti carbonara. You’re a client, and that cute lady is a server.

In the case of a chatbot, that’s not the case anymore. Look at the architecture:

PlantUML SVG diagram

First, a user posts a request to GitHub through a web user interface provided by GitHub. It is a communication hub for us. Then, the service connects to GitHub through its RESTful API and checks whether there are any new requests there. If something new is found, the service does the job, prepares a response, and posts it there. The client receives an email notification about a new response just posted to the ticket. The client then checks GitHub and finds the response.

badge

Here is how this would look in a restaurant: There would be a board with sticky notes. First, you write the note, “I’d like spaghetti carbonara with parmesan and fresh pepper on top” (Damn, I’m just too hungry now), and pin it to the board at number 15. Then, you return to your table. A chef from the kitchen checks that board and finds your sticky note. He makes that spaghetti, tops it with parmesan, fresh pepper, some basil leaves, and virgin olive oil … yeah, he makes it right … and puts it next to the board. You hear an announcement that order number 15 is ready. You go there, collect the food, return to your table, and enjoy.

The point is that there is no cute lady involved anymore. There is no server. There are two parties communicating with the board—you and the kitchen. The kitchen is our microservice, but it’s not a server anymore.

These two parties are perfectly decoupled now. They never talk to each other. And they both are clients of the communication hub, which is GitHub or a board in the restaurant.

Again, the microservice is not a server anymore. Instead, it is a client of a communication hub. And the flip of its position provides a lot of benefits to us, its developers.

No Need to Be Fast

badge

First of all, we don’t need to care much about the performance of our UI. Well, we don’t care at all, since we don’t have a UI. Do we care about the speed of responses on GitHub? Not really. When a user posts a message to GitHub, he or she doesn’t expect our chatbot to give an immediate answer in less than 100 milliseconds. (That’s what any properly designed web system must guarantee, I believe.)

We put a note on the board, and we assume that the kitchen is probably doing something else at the moment. We’ll wait for a few seconds or even minutes. If, on the other hand, I give an order to the waitress and she waits five seconds before replying back, I’ll be very surprised. If she keeps doing that with every question, I’ll start to wonder to myself if everything is OK with her.

I expect a user interface to be instant, while in a chat I have no problem allowing some time for the bot to answer. This happens naturally. We’re used to delays in chats, when we’re talking with real people. They need some time to process our information, to think, and to type something back.

But a user interface doesn’t have that luxury. It has to be bullet-fast; otherwise, I immediately get frustrated. The same thing happens to you, right?

No Need to Look Cute

badge

Another advantage of this no-server design is that there is no need to look pretty. There is no web interface, no HTML, no CSS, no graphic design. Perhaps not everybody really likes that. Most non-professional users may still prefer to talk to a cute server instead of sticking some paper notes to the board. But if we’re dealing with professional computer engineers, they’re not that demanding.

Rultor doesn’t have any web UI, and its users simply don’t know how it “looks.” It just talks to you. The only thing you see is its avatar in GitHub.

This saves a lot of money and time on design efforts, which are usually very expensive if you aim for high quality. If your web service looks average, most of its users will assume that it also works average. Many good ideas have simply died because their UI wasn’t as impressive as people were used to, thanks to all those Pinterests and Instagrams.

A good-looking server has a greater chance for bigger tips, right? If there is no server and we don’t see the chef, we judge him or her only by the quality of the food.

Same here. By getting rid of a UI, we allow ourselves to focus on the quality of the service we’re delivering. We don’t burn our time and money on being nice. We spend them on being useful.

Much Easier to Scale

badge

If we have too many stickies on that board, we just hire more cooks, or maybe even build another kitchen, and the problem is solved. We can handle as many customers as necessary. Well, as long as the board is powerful enough to handle multiple parallel users.

GitHub is a pretty big platform, with hundreds of thousands of users and projects. If we have too many requests coming in, we can just add more processing nodes to Rultor. Remember, we’re not a server anymore; we are a client of GitHub. We decide when to connect to GitHub and when to create responses to the requests submitted.

It is much easier to create a scalable client than a scalable server, mostly because there is nobody really waiting for us to respond quickly. The load of requests we’re getting can be managed much easier, since the decision of when to process them is made by us.

Mistakes Are Not So Visible

badge

When you’re standing in front of a customer, most of your mistakes are unforgivable, primarily because they are very visible. On the other hand, when you’re cooking something in the kitchen, nobody can see you and spot your faults. They will only spot them if the spaghetti has too much salt. In other words, they will judge you by your results, not by how you produce them.

It’s the same story with the microservice. When it works as a server, we expect it to be seamless, respond immediately, and present everything in a structured and organized way. If something goes wrong, it’s right here on the web page. Your best case is a 404, while the worst one is that you present some wrong information to the user. Even though the bug may not be critical inside the microservice engine, the user doesn’t know that. The user will judge you by your appearance and won’t forget even small mistakes.

However, when you both are clients of a message board, you don’t see each other. The user communicates with GitHub, and the microservice interacts with GitHub. Mistakes are less visible. Trust me, we have had many of them over the 18 months that Rultor has been in public use. We’ve had downtimes, we’ve had serious logical mistakes, and we’ve had data corruption. But very rarely have these problems become visible online. We merely saw them in our server logs. Users didn’t see them. Well, mostly :)

Everything Is Traceable

badge

Since there is a communication board between us, it’s very easy to see the entire history of our discussion, which is very intuitive. It’s like a Slack chat history. You see what we started from, who said what, and which conclusions were made.

Basically, you can’t have that visibility in a web UI. Well, you can probably create a special page with the “history of operations,” but who would check it? And how visible and simple would that information be? And, what’s most important, how would that information match up with the UI?

In the log, you’ll state that “the build was started,” but what’s the build and how was it started? How can I start it again? Using which buttons and web controls? It’s not clear.

Thus, the traceability of a chronological chat is unbeatable.

Easy to Integrate With Other Services

Yes, think about the future of this approach. If there is a centralized message board where users talk to a chatbot, why can’t other chatbots talk to each other too?

Forget about RESTful APIs. Just a message board where chatbots post their requests and collect responses. They are perfectly decoupled, replaceable, and very scalable. Also, their communication protocol is visible and very traceable. And they boast many other benefits, as was just explained above. It’s much more convenient for us, both users and programmers, to monitor them and create them.

Well, maybe it’s too extreme to get rid of RESTful APIs entirely, but to some extent, this approach is feasible, I believe.

I didn’t go too far with this idea, but something was done. We have a messaging platform that allows multiple chatbots to communicate with users. It’s called Netbout. It’s a very primitive web system with isolated discussions. Simply put, anyone can create a new discussion, invite a few friends, and post messages there. Both users and chatbots can do that.

So, when a new candidate wants to join Zerocracy, we ask that person to fill out an online form. When the candidate clicks the “Submit” button, a new discussion starts, and the first chatbot decides who should interview that person. The decision is made according to the skills listed in the form. The chatbot invites one of our best programmers to conduct the interview. When the interview is done, another chatbot explains to the candidate what the next steps are, registers him or her in our database, and starts to show the progress of work.

From a user perspective, it looks like he or she is talking to a few people who understand just a few simple commands. It is very intuitive and was easy to design.

I think chatbots are a good approach for interacting with microservices. Especially when users are more or less professional.

PS. Illustrations by Kristina Wheat.

© Yegor Bugayenko 2014–2018

What Do You Do With InterruptedException?

QR code

What Do You Do With InterruptedException?

  • Palo Alto, CA
  • comments

InterruptedException is a permanent source of pain in Java, for junior developers especially. But it shouldn’t be. It’s a rather simple and easy-to-understand idea. Let me try to describe and simplify it.

Crouching Tiger, Hidden Dragon (2000) by Ang Lee
Crouching Tiger, Hidden Dragon (2000) by Ang Lee

Let’s start with this code:

while (true) {
  // Nothing
}

What does it do? Nothing, it just spins the CPU endlessly. Can we terminate it? Not in Java. It will only stop when the entire JVM stops, when you hit Ctrl-C. There is no way in Java to terminate a thread unless the thread exits by itself. That’s the principle we have to have in mind, and everything else will just be obvious.

Let’s put this endless loop into a thread:

Thread loop = new Thread(
  new Runnable() {
    @Override
    public void run() {
      while (true) {
      }
    }
  }
);
loop.start();
// Now how do we stop it?

So, how do we stop a thread when we need it to stop?

Here is how it is designed in Java. There is a flag in every thread that we can set from the outside. And the thread may check it occasionally and stop its execution. Voluntarily! Here is how:

Thread loop = new Thread(
  new Runnable() {
    @Override
    public void run() {
      while (true) {
        if (Thread.interrupted()) {
          break;
        }
        // Continue to do nothing
      }
    }
  }
);
loop.start();
loop.interrupt();

This is the only way to ask a thread to stop. There are two methods that are used in this example. When I call loop.interrupt(), a flag is set to true somewhere inside the thread loop. When I call interrupted(), the flag is returned and immediately set to false. Yeah, that’s the design of the method. It checks the flag, returns it, and sets it to false. It’s ugly, I know.

Thus, if I never call Thread.interrupted() inside the thread and don’t exit when the flag is true, nobody will be able to stop me. Literally, I will just ignore their calls to interrupt(). They will ask me to stop, but I will ignore them. They won’t be able to interrupt me.

Thus, to summarize what we’ve learned so far, a properly designed thread will check that flag once in a while and stop gracefully. If the code doesn’t check the flag and never calls Thread.interrupted(), it accepts the fact that sooner or later it will be terminated cold turkey, by clicking Ctrl-C.

Sound logical so far? I hope so.

Now, there are some methods in JDK that check the flag for us and throw InterruptedException if it is set. For example, this is how the method Thread.sleep() is designed (taking a very primitive approach):

public static void sleep(long millis)
  throws InterruptedException {
  while (/* You still need to wait */) {
    if (Thread.interrupted()) {
      throw new InterruptedException();
    }
    // Keep waiting
  }
}

Why is it done this way? Why can’t it just wait and never check the flag? Well, I believe it’s done for a good reason. And the reason is the following (correct me if I’m wrong): The code should either be bullet-fast or interruption-ready, nothing in between.

If your code is fast, you never check the interruption flag, because you don’t want to deal with any interruptions. If your code is slow and may take seconds to execute, make it explicit and handle interruptions somehow.

That’s why InterruptedException is a checked exception. Its design tells you that if you want to pause for a few milliseconds, make your code interruption-ready. This is how it looks in practice:

try {
  Thread.sleep(100);
} catch (InterruptedException ex) {
  // Stop immediately and go home
}

Well, you could let it float up to a higher level, where they will be responsible for catching it. The point is that someone will have to catch it and do something with the thread. Ideally, just stop it, since that’s what the flag is about. If InterruptedException is thrown, it means someone checked the flag and our thread has to finish what it’s doing ASAP.

The owner of the thread doesn’t want to wait any longer. And we must respect the decision of our owner.

Thus, when you catch InterruptedException, you have to do whatever it takes to wrap up what you’re doing and exit.

Now, look again at the code of Thread.sleep():

public static void sleep(long millis)
  throws InterruptedException {
  while (/* ... */) {
    if (Thread.interrupted()) {
      throw new InterruptedException();
    }
  }
}

Remember, Thread.interrupted() not only returns the flag but also sets it to false. Thus, once InterruptedException is thrown, the flag is reset. The thread no longer knows anything about the interruption request sent by the owner.

The owner of the thread asked us to stop, Thread.sleep() detected that request, removed it, and threw InterruptedException. If you call Thread.sleep(), again, it will not know anything about that interruption request and will not throw anything.

See what I’m getting at? It’s very important not to lose that InterruptedException. We can’t just swallow it and move on. That would be a severe violation of the entire Java multi-threading idea. Our owner (the owner of our thread) is asking us to stop, and we just ignore it. That’s a very bad idea.

This is what most of us are doing with InterruptedException:

try {
  Thread.sleep(100);
} catch (InterruptedException ex) {
  throw new RuntimeException(ex);
}

It looks logical, but it doesn’t guarantee that the higher level will actually stop everything and exit. They may just catch a runtime exception there, and the thread will remain alive. The owner of the thread will be disappointed.

We have to inform the higher level that we just caught an interruption request. We can’t just throw a runtime exception. Such behavior would be too irresponsible. The entire thread received an interruption request, and we merely swallow it and convert it into a RuntimeException. We can’t treat such a serious situation so loosely.

This is what we have to do:

try {
  Thread.sleep(100);
} catch (InterruptedException ex) {
  Thread.currentThread().interrupt(); // Here!
  throw new RuntimeException(ex);
}

We’re setting the flag back to true!

Now, nobody will blame us for having an irresponsible attitude toward a valuable flag. We found it in true status, cleared it, set it back to true, and threw a runtime exception. What happens next, we don’t care.

I think that’s it. You can find a more detailed and official description of this problem here: Java Theory and Practice: Dealing With InterruptedException.

InterruptedException is a permanent source of pain in Java, for junior developers especially. But it shouldn’t be. It’s a rather simple and easy-to-understand idea. Let me try to describe and simplify it.

Crouching Tiger, Hidden Dragon (2000) by Ang Lee
Crouching Tiger, Hidden Dragon (2000) by Ang Lee

Let’s start with this code:

while (true) {
  // Nothing
}

What does it do? Nothing, it just spins the CPU endlessly. Can we terminate it? Not in Java. It will only stop when the entire JVM stops, when you hit Ctrl-C. There is no way in Java to terminate a thread unless the thread exits by itself. That’s the principle we have to have in mind, and everything else will just be obvious.

Let’s put this endless loop into a thread:

Thread loop = new Thread(
  new Runnable() {
    @Override
    public void run() {
      while (true) {
      }
    }
  }
);
loop.start();
// Now how do we stop it?

So, how do we stop a thread when we need it to stop?

Here is how it is designed in Java. There is a flag in every thread that we can set from the outside. And the thread may check it occasionally and stop its execution. Voluntarily! Here is how:

Thread loop = new Thread(
  new Runnable() {
    @Override
    public void run() {
      while (true) {
        if (Thread.interrupted()) {
          break;
        }
        // Continue to do nothing
      }
    }
  }
);
loop.start();
loop.interrupt();

This is the only way to ask a thread to stop. There are two methods that are used in this example. When I call loop.interrupt(), a flag is set to true somewhere inside the thread loop. When I call interrupted(), the flag is returned and immediately set to false. Yeah, that’s the design of the method. It checks the flag, returns it, and sets it to false. It’s ugly, I know.

Thus, if I never call Thread.interrupted() inside the thread and don’t exit when the flag is true, nobody will be able to stop me. Literally, I will just ignore their calls to interrupt(). They will ask me to stop, but I will ignore them. They won’t be able to interrupt me.

Thus, to summarize what we’ve learned so far, a properly designed thread will check that flag once in a while and stop gracefully. If the code doesn’t check the flag and never calls Thread.interrupted(), it accepts the fact that sooner or later it will be terminated cold turkey, by clicking Ctrl-C.

Sound logical so far? I hope so.

Now, there are some methods in JDK that check the flag for us and throw InterruptedException if it is set. For example, this is how the method Thread.sleep() is designed (taking a very primitive approach):

public static void sleep(long millis)
  throws InterruptedException {
  while (/* You still need to wait */) {
    if (Thread.interrupted()) {
      throw new InterruptedException();
    }
    // Keep waiting
  }
}

Why is it done this way? Why can’t it just wait and never check the flag? Well, I believe it’s done for a good reason. And the reason is the following (correct me if I’m wrong): The code should either be bullet-fast or interruption-ready, nothing in between.

If your code is fast, you never check the interruption flag, because you don’t want to deal with any interruptions. If your code is slow and may take seconds to execute, make it explicit and handle interruptions somehow.

That’s why InterruptedException is a checked exception. Its design tells you that if you want to pause for a few milliseconds, make your code interruption-ready. This is how it looks in practice:

try {
  Thread.sleep(100);
} catch (InterruptedException ex) {
  // Stop immediately and go home
}

Well, you could let it float up to a higher level, where they will be responsible for catching it. The point is that someone will have to catch it and do something with the thread. Ideally, just stop it, since that’s what the flag is about. If InterruptedException is thrown, it means someone checked the flag and our thread has to finish what it’s doing ASAP.

The owner of the thread doesn’t want to wait any longer. And we must respect the decision of our owner.

Thus, when you catch InterruptedException, you have to do whatever it takes to wrap up what you’re doing and exit.

Now, look again at the code of Thread.sleep():

public static void sleep(long millis)
  throws InterruptedException {
  while (/* ... */) {
    if (Thread.interrupted()) {
      throw new InterruptedException();
    }
  }
}

Remember, Thread.interrupted() not only returns the flag but also sets it to false. Thus, once InterruptedException is thrown, the flag is reset. The thread no longer knows anything about the interruption request sent by the owner.

The owner of the thread asked us to stop, Thread.sleep() detected that request, removed it, and threw InterruptedException. If you call Thread.sleep(), again, it will not know anything about that interruption request and will not throw anything.

See what I’m getting at? It’s very important not to lose that InterruptedException. We can’t just swallow it and move on. That would be a severe violation of the entire Java multi-threading idea. Our owner (the owner of our thread) is asking us to stop, and we just ignore it. That’s a very bad idea.

This is what most of us are doing with InterruptedException:

try {
  Thread.sleep(100);
} catch (InterruptedException ex) {
  throw new RuntimeException(ex);
}

It looks logical, but it doesn’t guarantee that the higher level will actually stop everything and exit. They may just catch a runtime exception there, and the thread will remain alive. The owner of the thread will be disappointed.

We have to inform the higher level that we just caught an interruption request. We can’t just throw a runtime exception. Such behavior would be too irresponsible. The entire thread received an interruption request, and we merely swallow it and convert it into a RuntimeException. We can’t treat such a serious situation so loosely.

This is what we have to do:

try {
  Thread.sleep(100);
} catch (InterruptedException ex) {
  Thread.currentThread().interrupt(); // Here!
  throw new RuntimeException(ex);
}

We’re setting the flag back to true!

Now, nobody will blame us for having an irresponsible attitude toward a valuable flag. We found it in true status, cleared it, set it back to true, and threw a runtime exception. What happens next, we don’t care.

I think that’s it. You can find a more detailed and official description of this problem here: Java Theory and Practice: Dealing With InterruptedException.

© Yegor Bugayenko 2014–2018

Vertical and Horizontal Decorating

QR code

Vertical and Horizontal Decorating

  • Moscow, Russia
  • comments

A decorator pattern is one of the best ways to add features to an object without changing its interface. I use composable decorators quite often and always question myself as to how to design them right when the list of features must be configurable. I’m not sure I have the right answer, but here is some food for thought.

The Apartment (1960) by Billy Wilder
The Apartment (1960) by Billy Wilder

Let’s say I have a list of numbers:

interface Numbers {
  Iterable<Integer> iterate();
}

Now I want to create a list that will only have odd, unique, positive, and sorted numbers. The first approach is vertical (I just made this name up):

Numbers numbers = new Sorted(
  new Unique(
    new Odds(
      new Positive(
        new ArrayNumbers(
          new Integer[] {
            -1, 78, 4, -34, 98, 4,
          }
        )
      )
    )
  )
);

The second approach is horizontal (again, a name I made up):

Numbers numbers = new Modified(
  new ArrayNumbers(
    new Integer[] {
      -1, 78, 4, -34, 98, 4,
    }
  ),
  new Diff[] {
    new Positive(),
    new Odds(),
    new Unique(),
    new Sorted(),
  }
);

See the difference? The first approach decorates ArrayNumbers “vertically,” adding functionality through the composable decorators Positive, Odds, Unique, and Sorted.

The second approach introduces the new interface Diff, which implements the core functionality of iterating numbers through instances of Positive, Odds, Unique, and Sorted:

interface Diff {
  Iterable<Integer> apply(Iterable<Integer> origin);
}

For the user of numbers, both approaches are the same. The difference is only in the design. Which one is better and when? It seems that vertical decorating is easier to implement and is more suitable for smaller objects that expose just a few methods.

As for my experience, I always tend to start with vertical decorating since it’s easier to implement but eventually migrate to a horizontal one when the number of decorators starts to grow.

A decorator pattern is one of the best ways to add features to an object without changing its interface. I use composable decorators quite often and always question myself as to how to design them right when the list of features must be configurable. I’m not sure I have the right answer, but here is some food for thought.

The Apartment (1960) by Billy Wilder
The Apartment (1960) by Billy Wilder

Let’s say I have a list of numbers:

interface Numbers {
  Iterable<Integer> iterate();
}

Now I want to create a list that will only have odd, unique, positive, and sorted numbers. The first approach is vertical (I just made this name up):

Numbers numbers = new Sorted(
  new Unique(
    new Odds(
      new Positive(
        new ArrayNumbers(
          new Integer[] {
            -1, 78, 4, -34, 98, 4,
          }
        )
      )
    )
  )
);

The second approach is horizontal (again, a name I made up):

Numbers numbers = new Modified(
  new ArrayNumbers(
    new Integer[] {
      -1, 78, 4, -34, 98, 4,
    }
  ),
  new Diff[] {
    new Positive(),
    new Odds(),
    new Unique(),
    new Sorted(),
  }
);

See the difference? The first approach decorates ArrayNumbers “vertically,” adding functionality through the composable decorators Positive, Odds, Unique, and Sorted.

The second approach introduces the new interface Diff, which implements the core functionality of iterating numbers through instances of Positive, Odds, Unique, and Sorted:

interface Diff {
  Iterable<Integer> apply(Iterable<Integer> origin);
}

For the user of numbers, both approaches are the same. The difference is only in the design. Which one is better and when? It seems that vertical decorating is easier to implement and is more suitable for smaller objects that expose just a few methods.

As for my experience, I always tend to start with vertical decorating since it’s easier to implement but eventually migrate to a horizontal one when the number of decorators starts to grow.

© Yegor Bugayenko 2014–2018

How to Set Up a Private Maven Repository in Amazon S3

QR code

How to Set Up a Private Maven Repository in Amazon S3

  • Kiev, Ukraine
  • comments

Amazon S3 is a perfect place for keeping private Maven artifacts. I assume you keep public artifacts in Maven Central because you want them to be available to everybody. Private artifacts are those you don’t want visible to anyone except members of your team. Thus, you want to deploy your .jar files there and make sure they are visible only by your team. Here is how we do this in all our Java projects.

Create an S3 Bucket

First, you create a new S3 bucket. I would recommend you name it using your project domain and a prefix. For example, with repo.teamed.io, repo is a prefix and teamed.io is the domain.

There’s no need to configure any permissions for this bucket. Just create it through the Amazon S3 console.

Create an IAM User

Create a new IAM user. I recommend you name it like teamed-maven if your project name is teamed.

Add a new “inline policy” to the user:

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::repo.teamed.io",
        "arn:aws:s3:::repo.teamed.io/*"
      ]
    }
  ]
}

Here, repo.teamed.io is the name of the S3 bucket you created a minute ago.

Make sure you have an “access key” for this new user. It must look similar to this:

key: AKIAI9NNNJD5D7X4TUVA
secret: t5tZQCwuaRhmlOXfbGE5aTBMFw34iFyxfCEr32av

The key is 20 characters (all caps), and the secret is 40 characters.

Extend settings.xml

Add this configuration to your ~/.m2/settings.xml file:

<settings>
  <servers>
    <server>
      <id>repo.teamed.io</id>
      <username>AKIAI9NNNJD5D7X4TUVA</username>
      <password>t5tZQCwuaRhmlOXfbGE5aTBMFw34iFyxfCEr32av</password>
    </server>
    [...]
  </servers>
  [...]
</settings>

Configure pom.xml

Add this configuration to pom.xml:

<project>
  <distributionManagement>
    <snapshotRepository>
      <id>repo.teamed.io</id>
      <url>s3://repo.teamed.io/snapshot</url>
    </snapshotRepository>
    <repository>
      <id>repo.teamed.io</id>
      <url>s3://repo.teamed.io/release</url>
    </repository>
  </distributionManagement>
  <repositories>
    <repository>
      <id>repo.teamed.io</id>
      <url>s3://repo.teamed.io/release</url>
    </repository>
  </repositories>
  [...]
</project>

Then, configure S3 Wagon, also in pom.xml:

<project>
  <build>
    <extensions>
      <extension>
        <groupId>org.kuali.maven.wagons</groupId>
        <artifactId>maven-s3-wagon</artifactId>
        <version>1.2.1</version>
      </extension>
    </extensions>
    [...]
  </build>
</project>

You’re ready to go. You can deploy your artifacts just by running Maven from the command line:

$ mvn clean deploy

Configure s3auth.com

Now you want to see these artifacts in your browser, in a secure mode, by providing secure credentials. I recommend you use s3auth.com, as explained in Basic HTTP Auth for S3 Buckets.

Configure Rultor

badge

Another recommendation is to configure rultor.com for deployment of your artifacts to S3 automatically.

First, encrypt your settings.xml with this Rultor remote:

$ gem install rultor
$ rultor encrypt -p me/test settings.xml

Instead of me/test, you should use the name of your GitHub project.

As a result, you will get a new file named settings.xml.asc. Add it to the root directory of your project, then commit and push. The file contains your secret information, but only the Rultor server can decrypt it.

Create a .rultor.yml file in the root directory of your project (The Rultor reference page explains this format in greater detail):

decrypt:
  settings.xml: "repo/settings.xml.asc"
deploy:
  script: |
    mvn clean deploy --settings ../settings.xml

Now it’s time to see how it all works together. Create a new ticket in the GitHub issue tracker and post something like this into it (read more about Rultor commands):

@rultor deploy

You will get a response in a few seconds. The rest will be done by Rultor.

That’s it.

Amazon S3 is a perfect place for keeping private Maven artifacts. I assume you keep public artifacts in Maven Central because you want them to be available to everybody. Private artifacts are those you don’t want visible to anyone except members of your team. Thus, you want to deploy your .jar files there and make sure they are visible only by your team. Here is how we do this in all our Java projects.

Create an S3 Bucket

First, you create a new S3 bucket. I would recommend you name it using your project domain and a prefix. For example, with repo.teamed.io, repo is a prefix and teamed.io is the domain.

There’s no need to configure any permissions for this bucket. Just create it through the Amazon S3 console.

Create an IAM User

Create a new IAM user. I recommend you name it like teamed-maven if your project name is teamed.

Add a new “inline policy” to the user:

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::repo.teamed.io",
        "arn:aws:s3:::repo.teamed.io/*"
      ]
    }
  ]
}

Here, repo.teamed.io is the name of the S3 bucket you created a minute ago.

Make sure you have an “access key” for this new user. It must look similar to this:

key: AKIAI9NNNJD5D7X4TUVA
secret: t5tZQCwuaRhmlOXfbGE5aTBMFw34iFyxfCEr32av

The key is 20 characters (all caps), and the secret is 40 characters.

Extend settings.xml

Add this configuration to your ~/.m2/settings.xml file:

<settings>
  <servers>
    <server>
      <id>repo.teamed.io</id>
      <username>AKIAI9NNNJD5D7X4TUVA</username>
      <password>t5tZQCwuaRhmlOXfbGE5aTBMFw34iFyxfCEr32av</password>
    </server>
    [...]
  </servers>
  [...]
</settings>

Configure pom.xml

Add this configuration to pom.xml:

<project>
  <distributionManagement>
    <snapshotRepository>
      <id>repo.teamed.io</id>
      <url>s3://repo.teamed.io/snapshot</url>
    </snapshotRepository>
    <repository>
      <id>repo.teamed.io</id>
      <url>s3://repo.teamed.io/release</url>
    </repository>
  </distributionManagement>
  <repositories>
    <repository>
      <id>repo.teamed.io</id>
      <url>s3://repo.teamed.io/release</url>
    </repository>
  </repositories>
  [...]
</project>

Then, configure S3 Wagon, also in pom.xml:

<project>
  <build>
    <extensions>
      <extension>
        <groupId>org.kuali.maven.wagons</groupId>
        <artifactId>maven-s3-wagon</artifactId>
        <version>1.2.1</version>
      </extension>
    </extensions>
    [...]
  </build>
</project>

You’re ready to go. You can deploy your artifacts just by running Maven from the command line:

$ mvn clean deploy

Configure s3auth.com

Now you want to see these artifacts in your browser, in a secure mode, by providing secure credentials. I recommend you use s3auth.com, as explained in Basic HTTP Auth for S3 Buckets.

Configure Rultor

badge

Another recommendation is to configure rultor.com for deployment of your artifacts to S3 automatically.

First, encrypt your settings.xml with this Rultor remote:

$ gem install rultor
$ rultor encrypt -p me/test settings.xml

Instead of me/test, you should use the name of your GitHub project.

As a result, you will get a new file named settings.xml.asc. Add it to the root directory of your project, then commit and push. The file contains your secret information, but only the Rultor server can decrypt it.

Create a .rultor.yml file in the root directory of your project (The Rultor reference page explains this format in greater detail):

decrypt:
  settings.xml: "repo/settings.xml.asc"
deploy:
  script: |
    mvn clean deploy --settings ../settings.xml

Now it’s time to see how it all works together. Create a new ticket in the GitHub issue tracker and post something like this into it (read more about Rultor commands):

@rultor deploy

You will get a response in a few seconds. The rest will be done by Rultor.

That’s it.

© Yegor Bugayenko 2014–2018

Redundant Variables Are Pure Evil

QR code

Redundant Variables Are Pure Evil

  • Kiev, Ukraine
  • comments

A redundant variable is one that exists exclusively to explain its value. I strongly believe that such a variable is not only pure noise but also evil, with a very negative effect on code readability. When we introduce a redundant variable, we intend to make our code cleaner and easier to read. In reality, though, we make it more verbose and difficult to understand. Without exception, any variable used only once is redundant and must be replaced with a value.

Y Tu Mamá También (2001) by Alfonso Cuarón
Y Tu Mamá También (2001) by Alfonso Cuarón

Here, variable fileName is redundant:

String fileName = "test.txt";
print("Length is " + new File(fileName).length());

This code must look differently:

print("Length is " + new File("test.txt").length());

This example is very primitive, but I’m sure you’ve seen these redundant variables many times. We use them to “explain” the code—it’s not just a string literal "test.txt" anymore but a fileName. The code looks easier to understand, right? Not really.

Let’s dig into what “readability” of code is in the first place. I think this quality can be measured by the number of seconds I need to understand the code I’m looking at. The longer the timeframe, the lower the readability. Ideally, I want to understand any piece of code in a few seconds. If I can’t, that’s a failure of its author.

Remember, if I don’t understand you, it’s your fault.

An increasing length of code degrades readability. So the more variable names I have to remember while reading through it, the longer it takes to digest the code and come to a conclusion about its purpose and effects. I think four is the maximum number of variables I can comfortably keep in my head without thinking about quitting the job.

New variables make the code longer because they need extra lines to be declared. And they make the code more complex because its reader has to remember more names.

Thus, when you want to introduce a new variable to explain what your code is doing, stop and think. Your code is too complex and long in the first place! Refactor it using new objects or methods but not variables. Make your code shorter by moving pieces of it into new classes or private methods.

Moreover, I think that in perfectly designed methods, you won’t need any variables aside from method arguments.

A redundant variable is one that exists exclusively to explain its value. I strongly believe that such a variable is not only pure noise but also evil, with a very negative effect on code readability. When we introduce a redundant variable, we intend to make our code cleaner and easier to read. In reality, though, we make it more verbose and difficult to understand. Without exception, any variable used only once is redundant and must be replaced with a value.

Y Tu Mamá También (2001) by Alfonso Cuarón
Y Tu Mamá También (2001) by Alfonso Cuarón

Here, variable fileName is redundant:

String fileName = "test.txt";
print("Length is " + new File(fileName).length());

This code must look differently:

print("Length is " + new File("test.txt").length());

This example is very primitive, but I’m sure you’ve seen these redundant variables many times. We use them to “explain” the code—it’s not just a string literal "test.txt" anymore but a fileName. The code looks easier to understand, right? Not really.

Let’s dig into what “readability” of code is in the first place. I think this quality can be measured by the number of seconds I need to understand the code I’m looking at. The longer the timeframe, the lower the readability. Ideally, I want to understand any piece of code in a few seconds. If I can’t, that’s a failure of its author.

Remember, if I don’t understand you, it’s your fault.

An increasing length of code degrades readability. So the more variable names I have to remember while reading through it, the longer it takes to digest the code and come to a conclusion about its purpose and effects. I think four is the maximum number of variables I can comfortably keep in my head without thinking about quitting the job.

New variables make the code longer because they need extra lines to be declared. And they make the code more complex because its reader has to remember more names.

Thus, when you want to introduce a new variable to explain what your code is doing, stop and think. Your code is too complex and long in the first place! Refactor it using new objects or methods but not variables. Make your code shorter by moving pieces of it into new classes or private methods.

Moreover, I think that in perfectly designed methods, you won’t need any variables aside from method arguments.

© Yegor Bugayenko 2014–2018

Why Many Return Statements Are a Bad Idea in OOP

QR code

Why Many Return Statements Are a Bad Idea in OOP

  • Kiev, Ukraine
  • comments

This debate is very old, but I have something to say too. The question is whether a method may have multiple return statements or always just one. The answer may surprise you: In a pure object-oriented world, a method must have a single return statement and nothing else. Yes, just a return statement and that’s it. No other operators or statements. Just return. All arguments in favor of multiple return statements go against the very idea of object-oriented programming.

This is a classical example:

public int max(int a, int b) {
  if (a > b) {
    return a;
  }
  return b;
}

The code above has two return statements, and it is shorter than this one with a single return:

public int max(int a, int b) {
  int m;
  if (a > b) {
    m = a;
  } else {
    m = b;
  }
  return m;
}

More verbose, less readable, and slower, right? Right.

This is the code in a pure object-oriented world:

public int max(int a, int b) {
  return new If(
    new GreaterThan(a, b),
    a, b
  );
}

What do you think now? There are no statements or operators. No if and no >. Instead, there are objects of class If and GreaterThan.

This is a pure and clean object-oriented approach.

However, Java doesn’t have that. Java (and many other pseudo OOP languages) gives us operators like if, else, switch, for, while, etc. instead of giving built-in classes, which would do the same. Because of that, we continue to think in terms of procedures and keep talking about whether two return statements are better than one.

If your code is truly object-oriented, you won’t be able to have more than one return. Moreover, you will have nothing except a return in each method. Actually, you will have only two operators in the entire software—new and return. That’s it.

Until we’re there, let’s stick with just one return and at least try to look like pure OOP.

This debate is very old, but I have something to say too. The question is whether a method may have multiple return statements or always just one. The answer may surprise you: In a pure object-oriented world, a method must have a single return statement and nothing else. Yes, just a return statement and that’s it. No other operators or statements. Just return. All arguments in favor of multiple return statements go against the very idea of object-oriented programming.

This is a classical example:

public int max(int a, int b) {
  if (a > b) {
    return a;
  }
  return b;
}

The code above has two return statements, and it is shorter than this one with a single return:

public int max(int a, int b) {
  int m;
  if (a > b) {
    m = a;
  } else {
    m = b;
  }
  return m;
}

More verbose, less readable, and slower, right? Right.

This is the code in a pure object-oriented world:

public int max(int a, int b) {
  return new If(
    new GreaterThan(a, b),
    a, b
  );
}

What do you think now? There are no statements or operators. No if and no >. Instead, there are objects of class If and GreaterThan.

This is a pure and clean object-oriented approach.

However, Java doesn’t have that. Java (and many other pseudo OOP languages) gives us operators like if, else, switch, for, while, etc. instead of giving built-in classes, which would do the same. Because of that, we continue to think in terms of procedures and keep talking about whether two return statements are better than one.

If your code is truly object-oriented, you won’t be able to have more than one return. Moreover, you will have nothing except a return in each method. Actually, you will have only two operators in the entire software—new and return. That’s it.

Until we’re there, let’s stick with just one return and at least try to look like pure OOP.

© Yegor Bugayenko 2014–2018

Checked vs. Unchecked Exceptions: The Debate Is Not Over

QR code

Checked vs. Unchecked Exceptions: The Debate Is Not Over

  • Sunnyvale, CA
  • comments

Do we need checked exceptions at all? The debate is over, isn’t it? Not for me. While most object-oriented languages don’t have them, and most programmers think checked exceptions are a Java mistake, I believe in the opposite—unchecked exceptions are the mistake. Moreover, I believe multiple exception types are a bad idea too.

True Romance (1993) by Tony Scott
True Romance (1993) by Tony Scott

Let me first explain how I understand exceptions in object-oriented programming. Then I’ll compare my understanding with a “traditional” approach, and we’ll discuss the differences. So, my understanding first.

Say there is a method that saves some binary data to a file:

public void save(File file, byte[] data)
  throws Exception {
  // save data to the file
}

When everything goes right, the method just saves the data and returns control. When something is wrong, it throws Exception and we have to do something about it:

try {
  save(file, data);
} catch (Exception ex) {
  System.out.println("Sorry, we can't save right now.");
}

When a method says it throws an exception, I understand that the method is not safe. It may fail sometimes, and it’s my responsibility to either 1) handle this failure or 2) declare myself as unsafe too.

I know each method is designed with a single responsibility principle in mind. This is a guarantee to me that if method save() fails, it means the entire saving operation can’t be completed. If I need to know what the cause of this failure was, I will un-chain the exception—traverse the stack of chained exceptions and stack traces encapsulated in ex.

I never use exceptions for flow control, which means I never recover situations where exceptions are thrown. When an exception occurs, I let it float up to the highest level of the application. Sometimes I rethrow it in order to add more semantic information to the chain. That’s why it doesn’t matter to me what the cause of the exception thrown by save() was. I just know the method failed. That’s enough for me. Always.

For the same reason, I don’t need to differentiate between different exception types. I just don’t need that type of hierarchy. Exception is enough for me. Again, that’s because I don’t use exceptions for flow control.

That’s how I understand exceptions.

According to this paradigm, I would say we must:

  • Always use checked exceptions.
  • Never throw/use unchecked exceptions.
  • Use only Exception, without any sub-types.
  • Always declare one exception type in the throws block.
  • Never catch without rethrowing; read more about that here.

This paradigm diverges from many other articles I’ve found on this subject. Let’s compare and discuss.

Runtime vs. API Exceptions

Oracle says some exceptions should be part of API (checked ones) while some are runtime exceptions and should not be part of it (unchecked). They will be documented in JavaDoc but not in the method signature.

I don’t understand the logic here, and I’m sure Java designers don’t understand it either. How and why are some exceptions important while others are not? Why do some of them deserve a proper API position in the throws block of the method signature while others don’t? What is the criteria?

I have an answer here, though. By introducing checked and unchecked exceptions, Java developers tried to solve the problem of methods that are too complex and messy. When a method is too big and does too many things at the same time (violates the single responsibility principle), it’s definitely better to let us keep some exceptions “hidden” (a.k.a. unchecked). But it’s not a real solution. It is only a temporary patch that does all of us more harm than good—methods keep growing in size and complexity.

Unchecked exceptions are a mistake in Java design, not checked ones.

Hiding the fact that a method may fail at some point is a mistake. That’s exactly what unchecked exceptions do.

Instead, we should make this fact visible. When a method does too many things, there will be too many points of failure, and the author of the method will realize that something is wrong—a method should not throw exceptions in so many situations. This will lead to refactoring. The existence of unchecked exceptions leads to a mess. By the way, checked exceptions don’t exist at all in Ruby, C#, Python, PHP, etc. This means that creators of these languages understand OOP even less than Java authors.

Checked Exceptions Are Too Noisy

Another common argument against checked exceptions is that they make our code more verbose. We have to put try/catch everywhere instead of staying focused on the main logic. Bozhidar Bozhanov even suggests a technical solution for this verbosity problem.

Again, I don’t understand this logic. If I want to do something when method save() fails, I catch the exception and handle the situation somehow. If I don’t want to do that, I just say my method also throws and pay no attention to exception handling. What is the problem? Where is the verbosity coming from?

I have an answer here, too. It’s coming from the existence of unchecked exceptions. We simply can’t always ignore failure, because the interfaces we’re using don’t allow us to do this. That’s all. For example, class Runnable, which is widely used for multi-thread programming, has method run() that is not supposed to throw anything. That’s why we always have to catch everything inside the method and rethrow checked exceptions as unchecked.

If all methods in all Java interfaces would be declared either as “safe” (throws nothing) or “unsafe” (throws Exception), everything would become logical and clear. If you want to stay “safe,” take responsibility for failure handling. Otherwise, be “unsafe” and let your users worry about safety.

No noise, very clean code, and obvious logic.

Inappropriately Exposed Implementation Details

Some say the ability to put a checked exception into throws in the method signature instead of catching it here and rethrowing a new type encourages us to have too many irrelevant exception types in method signatures. For example, our method save() may declare that it may throw OutOfMemoryException, even though it seems to have nothing to do with memory allocation. But it does allocate some memory, right? So such a memory overflow may happen during a file saving operation.

Yet again, I don’t get the logic of this argument. If all exceptions are checked, and we don’t have multiple exception types, we just throw Exception everywhere, and that’s it. Why do we need to care about the exception type in the first place? If we don’t use exceptions to control flow, we won’t do this.

If we really want to make our application memory overflow-resistant, we will introduce some memory manager, which will have something like the bigEnough() method, which will tell us whether our heap is big enough for the next operation. Using exceptions in such situations is a totally inappropriate approach to exception management in OOP.

Recoverable Exceptions

Joshua Bloch, in Effective Java, says to “use checked exceptions for recoverable conditions and runtime exceptions for programming errors.” He means something like this:

try {
  save(file, data);
} catch (Exception ex) {
  // We can't save the file, but it's OK
  // Let's move on and do something else
}

How is that any different from a famous anti-pattern called Don’t Use Exceptions for Flow Control? Joshua, with all due respect, you’re wrong. There are no such things as recoverable conditions in OOP. An exception indicates that the execution of a chain of calls from method to method is broken, and it’s time to go up through the chain and stop somewhere. But we never go back again after the exception:

App#run()
  Data#update()
    Data#write()
      File#save() <-- Boom, there's a failure here, so we go up

We can start this chain again, but we don’t go back after throw. In other words, we don’t do anything in the catch block. We only report the problem and wrap up execution. We never “recover!”


All arguments against checked exceptions demonstrate nothing but a serious misunderstanding of object-oriented programming by their authors. The mistake in Java and in many other languages is the existence of unchecked exceptions, not checked ones.

Do we need checked exceptions at all? The debate is over, isn’t it? Not for me. While most object-oriented languages don’t have them, and most programmers think checked exceptions are a Java mistake, I believe in the opposite—unchecked exceptions are the mistake. Moreover, I believe multiple exception types are a bad idea too.

True Romance (1993) by Tony Scott
True Romance (1993) by Tony Scott

Let me first explain how I understand exceptions in object-oriented programming. Then I’ll compare my understanding with a “traditional” approach, and we’ll discuss the differences. So, my understanding first.

Say there is a method that saves some binary data to a file:

public void save(File file, byte[] data)
  throws Exception {
  // save data to the file
}

When everything goes right, the method just saves the data and returns control. When something is wrong, it throws Exception and we have to do something about it:

try {
  save(file, data);
} catch (Exception ex) {
  System.out.println("Sorry, we can't save right now.");
}

When a method says it throws an exception, I understand that the method is not safe. It may fail sometimes, and it’s my responsibility to either 1) handle this failure or 2) declare myself as unsafe too.

I know each method is designed with a single responsibility principle in mind. This is a guarantee to me that if method save() fails, it means the entire saving operation can’t be completed. If I need to know what the cause of this failure was, I will un-chain the exception—traverse the stack of chained exceptions and stack traces encapsulated in ex.

I never use exceptions for flow control, which means I never recover situations where exceptions are thrown. When an exception occurs, I let it float up to the highest level of the application. Sometimes I rethrow it in order to add more semantic information to the chain. That’s why it doesn’t matter to me what the cause of the exception thrown by save() was. I just know the method failed. That’s enough for me. Always.

For the same reason, I don’t need to differentiate between different exception types. I just don’t need that type of hierarchy. Exception is enough for me. Again, that’s because I don’t use exceptions for flow control.

That’s how I understand exceptions.

According to this paradigm, I would say we must:

  • Always use checked exceptions.
  • Never throw/use unchecked exceptions.
  • Use only Exception, without any sub-types.
  • Always declare one exception type in the throws block.
  • Never catch without rethrowing; read more about that here.

This paradigm diverges from many other articles I’ve found on this subject. Let’s compare and discuss.

Runtime vs. API Exceptions

Oracle says some exceptions should be part of API (checked ones) while some are runtime exceptions and should not be part of it (unchecked). They will be documented in JavaDoc but not in the method signature.

I don’t understand the logic here, and I’m sure Java designers don’t understand it either. How and why are some exceptions important while others are not? Why do some of them deserve a proper API position in the throws block of the method signature while others don’t? What is the criteria?

I have an answer here, though. By introducing checked and unchecked exceptions, Java developers tried to solve the problem of methods that are too complex and messy. When a method is too big and does too many things at the same time (violates the single responsibility principle), it’s definitely better to let us keep some exceptions “hidden” (a.k.a. unchecked). But it’s not a real solution. It is only a temporary patch that does all of us more harm than good—methods keep growing in size and complexity.

Unchecked exceptions are a mistake in Java design, not checked ones.

Hiding the fact that a method may fail at some point is a mistake. That’s exactly what unchecked exceptions do.

Instead, we should make this fact visible. When a method does too many things, there will be too many points of failure, and the author of the method will realize that something is wrong—a method should not throw exceptions in so many situations. This will lead to refactoring. The existence of unchecked exceptions leads to a mess. By the way, checked exceptions don’t exist at all in Ruby, C#, Python, PHP, etc. This means that creators of these languages understand OOP even less than Java authors.

Checked Exceptions Are Too Noisy

Another common argument against checked exceptions is that they make our code more verbose. We have to put try/catch everywhere instead of staying focused on the main logic. Bozhidar Bozhanov even suggests a technical solution for this verbosity problem.

Again, I don’t understand this logic. If I want to do something when method save() fails, I catch the exception and handle the situation somehow. If I don’t want to do that, I just say my method also throws and pay no attention to exception handling. What is the problem? Where is the verbosity coming from?

I have an answer here, too. It’s coming from the existence of unchecked exceptions. We simply can’t always ignore failure, because the interfaces we’re using don’t allow us to do this. That’s all. For example, class Runnable, which is widely used for multi-thread programming, has method run() that is not supposed to throw anything. That’s why we always have to catch everything inside the method and rethrow checked exceptions as unchecked.

If all methods in all Java interfaces would be declared either as “safe” (throws nothing) or “unsafe” (throws Exception), everything would become logical and clear. If you want to stay “safe,” take responsibility for failure handling. Otherwise, be “unsafe” and let your users worry about safety.

No noise, very clean code, and obvious logic.

Inappropriately Exposed Implementation Details

Some say the ability to put a checked exception into throws in the method signature instead of catching it here and rethrowing a new type encourages us to have too many irrelevant exception types in method signatures. For example, our method save() may declare that it may throw OutOfMemoryException, even though it seems to have nothing to do with memory allocation. But it does allocate some memory, right? So such a memory overflow may happen during a file saving operation.

Yet again, I don’t get the logic of this argument. If all exceptions are checked, and we don’t have multiple exception types, we just throw Exception everywhere, and that’s it. Why do we need to care about the exception type in the first place? If we don’t use exceptions to control flow, we won’t do this.

If we really want to make our application memory overflow-resistant, we will introduce some memory manager, which will have something like the bigEnough() method, which will tell us whether our heap is big enough for the next operation. Using exceptions in such situations is a totally inappropriate approach to exception management in OOP.

Recoverable Exceptions

Joshua Bloch, in Effective Java, says to “use checked exceptions for recoverable conditions and runtime exceptions for programming errors.” He means something like this:

try {
  save(file, data);
} catch (Exception ex) {
  // We can't save the file, but it's OK
  // Let's move on and do something else
}

How is that any different from a famous anti-pattern called Don’t Use Exceptions for Flow Control? Joshua, with all due respect, you’re wrong. There are no such things as recoverable conditions in OOP. An exception indicates that the execution of a chain of calls from method to method is broken, and it’s time to go up through the chain and stop somewhere. But we never go back again after the exception:

App#run()
  Data#update()
    Data#write()
      File#save() <-- Boom, there's a failure here, so we go up

We can start this chain again, but we don’t go back after throw. In other words, we don’t do anything in the catch block. We only report the problem and wrap up execution. We never “recover!”


All arguments against checked exceptions demonstrate nothing but a serious misunderstanding of object-oriented programming by their authors. The mistake in Java and in many other languages is the existence of unchecked exceptions, not checked ones.

© Yegor Bugayenko 2014–2018

Public Static Literals ... Are Not a Solution for Data Duplication

QR code

Public Static Literals ... Are Not a Solution for Data Duplication

  • Palo Alto, CA
  • comments

I have a new String(array,"UTF-8") in one place and exactly the same code in another place in my app. Actually, I may have it in many places. And every time, I have to use that "UTF-8" constant in order to create a String from a byte array. It would be very convenient to define it once somewhere and reuse it, just like Apache Commons is doing; see CharEncoding.UTF_8 (There are many other static literals there). These guys are setting a bad example! public static “properties” are as bad as utility classes.

The Shining (1980) by Stanley Kubrick
The Shining (1980) by Stanley Kubrick

Here is what I’m talking about, specifically:

package org.apache.commons.lang3;
public class CharEncoding {
  public static final String UTF_8 = "UTF-8";
  // some other methods and properties
}

Now, when I need to create a String from a byte array, I use this:

import org.apache.commons.lang3.CharEncoding;
String text = new String(array, CharEncoding.UTF_8);

Let’s say I want to convert a String into a byte array:

import org.apache.commons.lang3.CharEncoding;
byte[] array = text.getBytes(CharEncoding.UTF_8);

Looks convenient, right? This is what the designers of Apache Commons think (one of the most popular but simply terrible libraries in the Java world). I encourage you to think differently. I can’t tell you to stop using Apache Commons, because we just don’t have a better alternative (yet!). But in your own code, don’t use public static properties—ever. Even if this code may look convenient to you, it’s a very bad design.

The reason why is very similar to utility classes with public static methods—they are unbreakable hard-coded dependencies. Once you use that CharEncoding.UTF_8, your object starts to depend on this data, and its user (the user of your object) can’t break this dependency. You may say that this is your intention, in the case of a "UTF-8" constant—to make sure that Unicode is specifically and exclusively being used. In this particular example, this may be true, but look at it from a more global perspective.

Let me show you the alternative I have in mind before we continue. Here is what I’m suggesting instead to convert a byte array into a String:

String text = new UTF8String(array);

It’s pseudo-code, since Java designers made class String final and we can’t really extend it and create UTF8String, but you get the idea. In the real world, this would look like this:

String text = new UTF8String(array).toString();

As you see, we encapsulate the “UTF-8” constant somewhere inside the class UTF8String, and its users have no idea how exactly this “byte array to string” conversion is happening.

By introducing UTF8String, we solved the problem of “UTF-8” literal duplication. But we did it in a proper object-oriented way—we encapsulated the functionality inside a class and let everybody instantiate its objects and use them. We resolved the problem of functionality duplication, not just data duplication.

Placing data into one shared place (CharEncoding.UTF_8) doesn’t really solve the duplication problem; it actually makes it worse, mostly because it encourages everybody to duplicate functionality using the same piece of shared data.

My point here is that every time you see that you have some data duplication in your application, start thinking about the functionality you’re duplicating. You will easily find the code that is repeated again and again. Make a new class for this code and place the data there, as a private property (or private static property). That’s how you will improve your design and truly get rid of duplication.

PS. You can use a method instead of a class, but not a static literal.

I have a new String(array,"UTF-8") in one place and exactly the same code in another place in my app. Actually, I may have it in many places. And every time, I have to use that "UTF-8" constant in order to create a String from a byte array. It would be very convenient to define it once somewhere and reuse it, just like Apache Commons is doing; see CharEncoding.UTF_8 (There are many other static literals there). These guys are setting a bad example! public static “properties” are as bad as utility classes.

The Shining (1980) by Stanley Kubrick
The Shining (1980) by Stanley Kubrick

Here is what I’m talking about, specifically:

package org.apache.commons.lang3;
public class CharEncoding {
  public static final String UTF_8 = "UTF-8";
  // some other methods and properties
}

Now, when I need to create a String from a byte array, I use this:

import org.apache.commons.lang3.CharEncoding;
String text = new String(array, CharEncoding.UTF_8);

Let’s say I want to convert a String into a byte array:

import org.apache.commons.lang3.CharEncoding;
byte[] array = text.getBytes(CharEncoding.UTF_8);

Looks convenient, right? This is what the designers of Apache Commons think (one of the most popular but simply terrible libraries in the Java world). I encourage you to think differently. I can’t tell you to stop using Apache Commons, because we just don’t have a better alternative (yet!). But in your own code, don’t use public static properties—ever. Even if this code may look convenient to you, it’s a very bad design.

The reason why is very similar to utility classes with public static methods—they are unbreakable hard-coded dependencies. Once you use that CharEncoding.UTF_8, your object starts to depend on this data, and its user (the user of your object) can’t break this dependency. You may say that this is your intention, in the case of a "UTF-8" constant—to make sure that Unicode is specifically and exclusively being used. In this particular example, this may be true, but look at it from a more global perspective.

Let me show you the alternative I have in mind before we continue. Here is what I’m suggesting instead to convert a byte array into a String:

String text = new UTF8String(array);

It’s pseudo-code, since Java designers made class String final and we can’t really extend it and create UTF8String, but you get the idea. In the real world, this would look like this:

String text = new UTF8String(array).toString();

As you see, we encapsulate the “UTF-8” constant somewhere inside the class UTF8String, and its users have no idea how exactly this “byte array to string” conversion is happening.

By introducing UTF8String, we solved the problem of “UTF-8” literal duplication. But we did it in a proper object-oriented way—we encapsulated the functionality inside a class and let everybody instantiate its objects and use them. We resolved the problem of functionality duplication, not just data duplication.

Placing data into one shared place (CharEncoding.UTF_8) doesn’t really solve the duplication problem; it actually makes it worse, mostly because it encourages everybody to duplicate functionality using the same piece of shared data.

My point here is that every time you see that you have some data duplication in your application, start thinking about the functionality you’re duplicating. You will easily find the code that is repeated again and again. Make a new class for this code and place the data there, as a private property (or private static property). That’s how you will improve your design and truly get rid of duplication.

PS. You can use a method instead of a class, but not a static literal.

© Yegor Bugayenko 2014–2018

XML Data and XSL Views in Takes Framework

QR code

XML Data and XSL Views in Takes Framework

  • Palo Alto, CA
  • comments

badge

A year ago, I tried to explain how effectively data and its presentation can be separated in a web application with the help of XML and XSL. In a few words, instead of using templating (like JSP, Velocity, FreeMarker, etc.) and injection of data into HTML, we compose them in the form of an XML document and then let the XSL stylesheet transform them into HTML. Here is a brief example of how all this can be used together with the Takes framework.

First, let’s agree that templating is a bad idea in the first place. Yes, I mean it. The entire design of JSP is wrong, with all due respect to its creators. Here is how it works: Let’s say my website has to fetch the current exchange rate of the euro from a database and show it on the home page. Here’s how my index.jsp would look:

<html>
  <body>
    <p>EUR/USD: <%= rates.get() %></p>
  </body>
</html>

In order to create HTML, the JSP engine will have to call get() on object rates and render what’s returned through toString(). It’s a terrible design for a few reasons. First, the view is tightly coupled with the model. Second, the flexibility of rendering is very limited. Third, the result of rendering is not reusable, and views are not stackable. There are many other reasons … more about them in one of the next articles.

Let’s see how this should be done right. First, we let our model generate the output in XML format, for example:

<?xml version="1.1"?>
<page>
  <rate>1.1324</rate>
</page>

This is what the model will produce, having no knowledge of the view. Then, we create the view as an XSL stylesheet, which will transform XML into HTML:

<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns="http://www.w3.org/1999/xhtml">
  <xsl:template match="page">
    <html>
      <body>
        <p>
          <xsl:text>EUR/USD: </xsl:text>
          <xsl:value-of select="rate"/>
        </p>
      </body>
    </html>
  </xsl:template>
</xsl:stylesheet>

As you see, the view doesn’t know anything about the model in terms of implementation. All it knows is the format of XML data output produced by the model. Here is how you design it in the Takes framework. Let’s start with a simple example:

import org.takes.http.Exit;
import org.takes.http.FtCli;
public final class Entry {
  public static void main(final String... args) throws Exception {
    new FtCli(new TkApp(), args).start(Exit.NEVER);
  }
}

It’s a simple web application that starts a web server and never ends (it waits for connections in daemon mode). To make it work, we should create a simple “take” named TkApp:

import org.takes.Take;
import org.takes.tk.TkWrap;
final class TkApp extends TkWrap {
  @Override
  public Response act() {
    return new RsWithType(
      new RsText(
        "<page><rate>1.1324</rate></page>"
      ),
      "application/xml"
    );
  }
}

This “take” always returns the same XML response, but it doesn’t do any XSL transformation yet. We need to add the RsXSLT class to the picture:

@Override
public Response act() {
  return new RsXSLT(
    new RsWithType(
      new RsText(
        "<?xml-stylesheet type='text/xsl' href='/xsl/index.xsl'?>"
        + "<page><rate>1.1324</rate></page>"
      ),
      "application/xml"
    )
  );
}

Excuse me for using string concatenation, which is a bad practice; it’s merely for the simplicity of the example.

As you see, I also added an XML stylesheet processing instruction to the XML. RsXSLT will understand it and try to find the /xsl/index.xsl resource on classpath. You see the content of that file above.

That’s it.

Well, not really. Building XML from strings is not a good idea. We have a better instrument in the Takes framework. We use Xembly, which is a simple imperative language for building and modifying XML documents. More about it here: Xembly, an Assembly for XML.

Here is how our TkApp would look:

@Override
public Response act() {
  return new RsXSLT(
    new RsWithType(
      new RsXembly(
        new XeChain(
          new XeStylesheet("/xsl/index.xsl"),
          new XeAppend(
            "page",
            new XeDirectives(
              new Directives().add("rate").set("1.1324")
            )
          )
        )
      ),
      "application/xml"
    )
  );
}

The most important class here is RsXembly. The idea is to let model classes expose their data through Xembly “directives,” which will later be applied to a DOM structure by RsXembly.

XeChain, XeStylesheet, XeAppend, and XeDirectives expose directives but with different meanings (they are all instances of an XeSource interface). Their names describe their intentions rather well. XeChain just chains everything that is delivered by encapsulated “directive sources.” XeStylesheet returns directives that create a single XML processing instruction. XeAppend creates an XML node and adds encapsulated directives to it. XeDirectives simply returns what’s inside.

In the end, this code will create exactly the same XML document as I created above with string concatenation.

The beauty of this approach is in the perfect decoupling of data generation and XML building and translation between XML and HTML. It is perfectly reusable and “stackable.” We can transform the data in XML format multiple times, applying different XSL stylesheets to each one. We can even transform them into JSON without changing a line of code in model classes.

Moreover, we can format them differently, using rather powerful XSLT 2.0 instruments. XSLT by itself is a powerful and purely functional language that enables any possible data manipulations. No templating engine is even close to it.

Take a look at how it works in the RsPage class in Rultor, for example.

badge

A year ago, I tried to explain how effectively data and its presentation can be separated in a web application with the help of XML and XSL. In a few words, instead of using templating (like JSP, Velocity, FreeMarker, etc.) and injection of data into HTML, we compose them in the form of an XML document and then let the XSL stylesheet transform them into HTML. Here is a brief example of how all this can be used together with the Takes framework.

First, let’s agree that templating is a bad idea in the first place. Yes, I mean it. The entire design of JSP is wrong, with all due respect to its creators. Here is how it works: Let’s say my website has to fetch the current exchange rate of the euro from a database and show it on the home page. Here’s how my index.jsp would look:

<html>
  <body>
    <p>EUR/USD: <%= rates.get() %></p>
  </body>
</html>

In order to create HTML, the JSP engine will have to call get() on object rates and render what’s returned through toString(). It’s a terrible design for a few reasons. First, the view is tightly coupled with the model. Second, the flexibility of rendering is very limited. Third, the result of rendering is not reusable, and views are not stackable. There are many other reasons … more about them in one of the next articles.

Let’s see how this should be done right. First, we let our model generate the output in XML format, for example:

<?xml version="1.1"?>
<page>
  <rate>1.1324</rate>
</page>

This is what the model will produce, having no knowledge of the view. Then, we create the view as an XSL stylesheet, which will transform XML into HTML:

<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns="http://www.w3.org/1999/xhtml">
  <xsl:template match="page">
    <html>
      <body>
        <p>
          <xsl:text>EUR/USD: </xsl:text>
          <xsl:value-of select="rate"/>
        </p>
      </body>
    </html>
  </xsl:template>
</xsl:stylesheet>

As you see, the view doesn’t know anything about the model in terms of implementation. All it knows is the format of XML data output produced by the model. Here is how you design it in the Takes framework. Let’s start with a simple example:

import org.takes.http.Exit;
import org.takes.http.FtCli;
public final class Entry {
  public static void main(final String... args) throws Exception {
    new FtCli(new TkApp(), args).start(Exit.NEVER);
  }
}

It’s a simple web application that starts a web server and never ends (it waits for connections in daemon mode). To make it work, we should create a simple “take” named TkApp:

import org.takes.Take;
import org.takes.tk.TkWrap;
final class TkApp extends TkWrap {
  @Override
  public Response act() {
    return new RsWithType(
      new RsText(
        "<page><rate>1.1324</rate></page>"
      ),
      "application/xml"
    );
  }
}

This “take” always returns the same XML response, but it doesn’t do any XSL transformation yet. We need to add the RsXSLT class to the picture:

@Override
public Response act() {
  return new RsXSLT(
    new RsWithType(
      new RsText(
        "<?xml-stylesheet type='text/xsl' href='/xsl/index.xsl'?>"
        + "<page><rate>1.1324</rate></page>"
      ),
      "application/xml"
    )
  );
}

Excuse me for using string concatenation, which is a bad practice; it’s merely for the simplicity of the example.

As you see, I also added an XML stylesheet processing instruction to the XML. RsXSLT will understand it and try to find the /xsl/index.xsl resource on classpath. You see the content of that file above.

That’s it.

Well, not really. Building XML from strings is not a good idea. We have a better instrument in the Takes framework. We use Xembly, which is a simple imperative language for building and modifying XML documents. More about it here: Xembly, an Assembly for XML.

Here is how our TkApp would look:

@Override
public Response act() {
  return new RsXSLT(
    new RsWithType(
      new RsXembly(
        new XeChain(
          new XeStylesheet("/xsl/index.xsl"),
          new XeAppend(
            "page",
            new XeDirectives(
              new Directives().add("rate").set("1.1324")
            )
          )
        )
      ),
      "application/xml"
    )
  );
}

The most important class here is RsXembly. The idea is to let model classes expose their data through Xembly “directives,” which will later be applied to a DOM structure by RsXembly.

XeChain, XeStylesheet, XeAppend, and XeDirectives expose directives but with different meanings (they are all instances of an XeSource interface). Their names describe their intentions rather well. XeChain just chains everything that is delivered by encapsulated “directive sources.” XeStylesheet returns directives that create a single XML processing instruction. XeAppend creates an XML node and adds encapsulated directives to it. XeDirectives simply returns what’s inside.

In the end, this code will create exactly the same XML document as I created above with string concatenation.

The beauty of this approach is in the perfect decoupling of data generation and XML building and translation between XML and HTML. It is perfectly reusable and “stackable.” We can transform the data in XML format multiple times, applying different XSL stylesheets to each one. We can even transform them into JSON without changing a line of code in model classes.

Moreover, we can format them differently, using rather powerful XSLT 2.0 instruments. XSLT by itself is a powerful and purely functional language that enables any possible data manipulations. No templating engine is even close to it.

Take a look at how it works in the RsPage class in Rultor, for example.

© Yegor Bugayenko 2014–2018

There Can Be Only One Primary Constructor

QR code

There Can Be Only One Primary Constructor

  • Mountain View, CA
  • comments

I suggest classifying class constructors in OOP as primary and secondary. A primary constructor is the one that constructs an object and encapsulates other objects inside it. A secondary one is simply a preparation step before calling a primary constructor and is not really a constructor but rather an introductory layer in front of a real constructing mechanism.

The Matrix (1999) by The Wachowski Brothers
The Matrix (1999) by The Wachowski Brothers

Here is what I mean:

final class Cash {
  private final int cents;
  private final String currency;
  public Cash() { // secondary
    this(0);
  }
  public Cash(int cts) { // secondary
    this(cts, "USD");
  }
  public Cash(int cts, String crn) { // primary
    this.cents = cts;
    this.currency = crn;
  }
  // methods here
}

There are three constructors in the class—only one is primary and the other two are secondary. My definition of a secondary constructor is simple: It doesn’t do anything besides calling a primary constructor, through this(..).

My point here is that a properly designed class must have only one primary constructor, and it should be declared after all secondary ones. Why? There is only one reason behind this rule: It helps eliminate code duplication.

Without such a rule, we may have this design for our class:

final class Cash {
  private final int cents;
  private final String currency;
  public Cash() { // primary
    this.cents = 0;
    this.currency = "USD";
  }
  public Cash(int cts) { // primary
    this.cents = cts;
    this.currency = "USD";
  }
  public Cash(int cts, String crn) { // primary
    this.cents = cts;
    this.currency = crn;
  }
  // methods here
}

There’s not a lot of code here, but the duplication is massive and ugly; I hope you see it for yourself.

By strictly following this suggested rule, all classes will have a single entry point (point of construction), which is a primary constructor, and it will always be easy to find because it stays below all secondary constructors.

More about this subject in Elegant Objects, Section 1.2.

I suggest classifying class constructors in OOP as primary and secondary. A primary constructor is the one that constructs an object and encapsulates other objects inside it. A secondary one is simply a preparation step before calling a primary constructor and is not really a constructor but rather an introductory layer in front of a real constructing mechanism.

The Matrix (1999) by The Wachowski Brothers
The Matrix (1999) by The Wachowski Brothers

Here is what I mean:

final class Cash {
  private final int cents;
  private final String currency;
  public Cash() { // secondary
    this(0);
  }
  public Cash(int cts) { // secondary
    this(cts, "USD");
  }
  public Cash(int cts, String crn) { // primary
    this.cents = cts;
    this.currency = crn;
  }
  // methods here
}

There are three constructors in the class—only one is primary and the other two are secondary. My definition of a secondary constructor is simple: It doesn’t do anything besides calling a primary constructor, through this(..).

My point here is that a properly designed class must have only one primary constructor, and it should be declared after all secondary ones. Why? There is only one reason behind this rule: It helps eliminate code duplication.

Without such a rule, we may have this design for our class:

final class Cash {
  private final int cents;
  private final String currency;
  public Cash() { // primary
    this.cents = 0;
    this.currency = "USD";
  }
  public Cash(int cts) { // primary
    this.cents = cts;
    this.currency = "USD";
  }
  public Cash(int cts, String crn) { // primary
    this.cents = cts;
    this.currency = crn;
  }
  // methods here
}

There’s not a lot of code here, but the duplication is massive and ugly; I hope you see it for yourself.

By strictly following this suggested rule, all classes will have a single entry point (point of construction), which is a primary constructor, and it will always be easy to find because it stays below all secondary constructors.

More about this subject in Elegant Objects, Section 1.2.

© Yegor Bugayenko 2014–2018

A Few Thoughts on Unit Test Scaffolding

QR code

A Few Thoughts on Unit Test Scaffolding

  • Atherton, CA
  • comments

When I start to repeat myself in unit test methods by creating the same objects and preparing the data to run the test, I feel disappointed in my design. Long test methods with a lot of code duplication just don’t look right. To simplify and shorten them, there are basically two options, at least in Java: 1) private properties initialized through @Before and @BeforeClass, and 2) private static methods. They both look anti-OOP to me, and I think there is an alternative. Let me explain.

Léon: The Professional by Luc Besson
Léon: The Professional by Luc Besson

JUnit officially suggests a test fixture:

public final class MetricsTest {
  private File temp;
  private Folder folder;
  @Before
  public void prepare() {
    this.temp = Files.createTempDirectory("test");
    this.folder = new DiscFolder(this.temp);
    this.folder.save("first.txt", "Hello, world!");
    this.folder.save("second.txt", "Goodbye!");
  }
  @After
  public void clean() {
    FileUtils.deleteDirectory(this.temp);
  }
  @Test
  public void calculatesTotalSize() {
    assertEquals(22, new Metrics(this.folder).size());
  }
  @Test
  public void countsWordsInFiles() {
    assertEquals(4, new Metrics(this.folder).wc());
  }
}

I think it’s obvious what this test is doing. First, in prepare(), it creates a “test fixture” of type Folder. That is used in all three tests as an argument for the Metrics constructor. The real class being tested here is Metrics while this.folder is something we need in order to test it.

What’s wrong with this test? There is one serious issue: coupling between test methods. Test methods (and all tests in general) must be perfectly isolated from each other. This means that changing one test must not affect any others. In this example, that is not the case. When I want to change the countsWords() test, I have to change the internals of before(), which will affect the other method in the test “class.”

With all due respect to JUnit, the idea of creating test fixtures in @Before and @After is wrong, mostly because it encourages developers to couple test methods.

Here is how we can improve our test and isolate test methods:

public final class MetricsTest {
  @Test
  public void calculatesTotalSize() {
    final File dir = Files.createTempDirectory("test-1");
    final Folder folder = MetricsTest.folder(
      dir,
      "first.txt:Hello, world!",
      "second.txt:Goodbye!"
    );
    try {
      assertEquals(22, new Metrics(folder).size());
    } finally {
      FileUtils.deleteDirectory(dir);
    }
  }
  @Test
  public void countsWordsInFiles() {
    final File dir = Files.createTempDirectory("test-2");
    final Folder folder = MetricsTest.folder(
      dir,
      "alpha.txt:Three words here",
      "beta.txt:two words"
      "gamma.txt:one!"
    );
    try {
      assertEquals(6, new Metrics(folder).wc());
    } finally {
      FileUtils.deleteDirectory(dir);
    }
  }
  private static Folder folder(File dir, String... parts) {
    Folder folder = new DiscFolder(dir);
    for (final String part : parts) {
      final String[] pair = part.split(":", 2);
      this.folder.save(pair[0], pair[1]);
    }
    return folder;
  }
}

Does it look better now? We’re not there yet, but now our test methods are perfectly isolated. If I want to change one of them, I’m not going to affect the others because I pass all configuration parameters to a private static utility (!) method folder().

A utility method, huh? Yes, it smells.

The main issue with this design, even though it is way better than the previous one, is that it doesn’t prevent code duplication between test “classes.” If I need a similar test fixture of type Folder in another test case, I will have to move this static method there. Or even worse, I will have to create a utility class. Yes, there is nothing worse in object-oriented programming than utility classes.

A much better design would be to use “fake” objects instead of private static utilities. Here is how. First, we create a fake class and place it into src/main/java. This class can be used in tests and also in production code, if necessary (Fk for “fake”):

public final class FkFolder implements Folder, Closeable {
  private final File dir;
  private final String[] parts;
  public FkFolder(String... prts) {
    this(Files.createTempDirectory("test-1"), parts);
  }
  public FkFolder(File file, String... prts) {
    this.dir = file;
    this.parts = parts;
  }
  @Override
  public Iterable<File> files() {
    final Folder folder = new DiscFolder(this.dir);
    for (final String part : this.parts) {
      final String[] pair = part.split(":", 2);
      folder.save(pair[0], pair[1]);
    }
    return folder.files();
  }
  @Override
  public void close() {
    FileUtils.deleteDirectory(this.dir);
  }
}

Here is how our test will look now:

public final class MetricsTest {
  @Test
  public void calculatesTotalSize() {
    final String[] parts = {
      "first.txt:Hello, world!",
      "second.txt:Goodbye!"
    };
    try (final Folder folder = new FkFolder(parts)) {
      assertEquals(22, new Metrics(folder).size());
    }
  }
  @Test
  public void countsWordsInFiles() {
    final String[] parts = {
      "alpha.txt:Three words here",
      "beta.txt:two words"
      "gamma.txt:one!"
    };
    try (final Folder folder = new FkFolder(parts)) {
      assertEquals(6, new Metrics(folder).wc());
    }
  }
}

What do you think? Isn’t it better than what JUnit offers? Isn’t it more reusable and extensible than utility methods?

To summarize, I believe scaffolding in unit testing must be done through fake objects that are shipped together with production code.

When I start to repeat myself in unit test methods by creating the same objects and preparing the data to run the test, I feel disappointed in my design. Long test methods with a lot of code duplication just don’t look right. To simplify and shorten them, there are basically two options, at least in Java: 1) private properties initialized through @Before and @BeforeClass, and 2) private static methods. They both look anti-OOP to me, and I think there is an alternative. Let me explain.

Léon: The Professional by Luc Besson
Léon: The Professional by Luc Besson

JUnit officially suggests a test fixture:

public final class MetricsTest {
  private File temp;
  private Folder folder;
  @Before
  public void prepare() {
    this.temp = Files.createTempDirectory("test");
    this.folder = new DiscFolder(this.temp);
    this.folder.save("first.txt", "Hello, world!");
    this.folder.save("second.txt", "Goodbye!");
  }
  @After
  public void clean() {
    FileUtils.deleteDirectory(this.temp);
  }
  @Test
  public void calculatesTotalSize() {
    assertEquals(22, new Metrics(this.folder).size());
  }
  @Test
  public void countsWordsInFiles() {
    assertEquals(4, new Metrics(this.folder).wc());
  }
}

I think it’s obvious what this test is doing. First, in prepare(), it creates a “test fixture” of type Folder. That is used in all three tests as an argument for the Metrics constructor. The real class being tested here is Metrics while this.folder is something we need in order to test it.

What’s wrong with this test? There is one serious issue: coupling between test methods. Test methods (and all tests in general) must be perfectly isolated from each other. This means that changing one test must not affect any others. In this example, that is not the case. When I want to change the countsWords() test, I have to change the internals of before(), which will affect the other method in the test “class.”

With all due respect to JUnit, the idea of creating test fixtures in @Before and @After is wrong, mostly because it encourages developers to couple test methods.

Here is how we can improve our test and isolate test methods:

public final class MetricsTest {
  @Test
  public void calculatesTotalSize() {
    final File dir = Files.createTempDirectory("test-1");
    final Folder folder = MetricsTest.folder(
      dir,
      "first.txt:Hello, world!",
      "second.txt:Goodbye!"
    );
    try {
      assertEquals(22, new Metrics(folder).size());
    } finally {
      FileUtils.deleteDirectory(dir);
    }
  }
  @Test
  public void countsWordsInFiles() {
    final File dir = Files.createTempDirectory("test-2");
    final Folder folder = MetricsTest.folder(
      dir,
      "alpha.txt:Three words here",
      "beta.txt:two words"
      "gamma.txt:one!"
    );
    try {
      assertEquals(6, new Metrics(folder).wc());
    } finally {
      FileUtils.deleteDirectory(dir);
    }
  }
  private static Folder folder(File dir, String... parts) {
    Folder folder = new DiscFolder(dir);
    for (final String part : parts) {
      final String[] pair = part.split(":", 2);
      this.folder.save(pair[0], pair[1]);
    }
    return folder;
  }
}

Does it look better now? We’re not there yet, but now our test methods are perfectly isolated. If I want to change one of them, I’m not going to affect the others because I pass all configuration parameters to a private static utility (!) method folder().

A utility method, huh? Yes, it smells.

The main issue with this design, even though it is way better than the previous one, is that it doesn’t prevent code duplication between test “classes.” If I need a similar test fixture of type Folder in another test case, I will have to move this static method there. Or even worse, I will have to create a utility class. Yes, there is nothing worse in object-oriented programming than utility classes.

A much better design would be to use “fake” objects instead of private static utilities. Here is how. First, we create a fake class and place it into src/main/java. This class can be used in tests and also in production code, if necessary (Fk for “fake”):

public final class FkFolder implements Folder, Closeable {
  private final File dir;
  private final String[] parts;
  public FkFolder(String... prts) {
    this(Files.createTempDirectory("test-1"), parts);
  }
  public FkFolder(File file, String... prts) {
    this.dir = file;
    this.parts = parts;
  }
  @Override
  public Iterable<File> files() {
    final Folder folder = new DiscFolder(this.dir);
    for (final String part : this.parts) {
      final String[] pair = part.split(":", 2);
      folder.save(pair[0], pair[1]);
    }
    return folder.files();
  }
  @Override
  public void close() {
    FileUtils.deleteDirectory(this.dir);
  }
}

Here is how our test will look now:

public final class MetricsTest {
  @Test
  public void calculatesTotalSize() {
    final String[] parts = {
      "first.txt:Hello, world!",
      "second.txt:Goodbye!"
    };
    try (final Folder folder = new FkFolder(parts)) {
      assertEquals(22, new Metrics(folder).size());
    }
  }
  @Test
  public void countsWordsInFiles() {
    final String[] parts = {
      "alpha.txt:Three words here",
      "beta.txt:two words"
      "gamma.txt:one!"
    };
    try (final Folder folder = new FkFolder(parts)) {
      assertEquals(6, new Metrics(folder).wc());
    }
  }
}

What do you think? Isn’t it better than what JUnit offers? Isn’t it more reusable and extensible than utility methods?

To summarize, I believe scaffolding in unit testing must be done through fake objects that are shipped together with production code.

© Yegor Bugayenko 2014–2018

How Cookie-Based Authentication Works in the Takes Framework

QR code

How Cookie-Based Authentication Works in the Takes Framework

badge

When you enter your email and password into the Facebook login page, you get into your account. Then, wherever you go in the site, you always see your photo at the top right corner of the page. Facebook remembers you and doesn’t ask for the password again and again. This works thanks to HTTP cookies and is called cookie-based authentication. Even though this mechanism often causes some security problems, it is very popular and simple. Here is how Takes makes it possible in a few lines of code.

First, let’s see how it works. Moreover, let’s see how I believe it should work.

Step one: The user enters an email and password and clicks “submit.” The server receives a POST request with this information inside:

POST / HTTP/1.1
Host: www.facebook.com
Content-Type: application/x-www-form-urlencoded

email=me@yegor256.com&password=itisasecret

The server matches the provided information with its records and decides what to do. If the information is invalid, it returns the same login page, asking you to enter it all again. If the information is valid, the server returns something like this:

HTTP/1.1 303 See Other
Location: www.facebook.com
Set-Cookie: user=me@yegor256.com

Since the response status code is 303, the browser goes to the page specified in the Location header and opens the front page of the site. This is what it sends to the server:

GET / HTTP/1.1
Host: www.facebook.com
Cookie: user=me@yegor256.com

The server gets my email from the Cookie header and understands that it’s me again! No need to ask for the password once more. The server trusts the information from the cookie. That’s it. That’s what cookie-based authentication is all about.

Wait … What About Security?

Right, what about security? If the server trusts any browser request with a user email in the Cookie header, anyone would be able to send my email from another place and get access to my account.

The first step to prevent this is to encrypt the email with a secret encryption key, known only to the server. Nobody except the server itself will be able to encrypt it the same way the server needs to decrypt it. The response would look like this, using an example of encryption by XOR cipher with bamboo as a secret key:

HTTP/1.1 303 See Other
Location: www.facebook.com
Set-Cookie: user=b1ccafd92c568515100f5c4d104671003cfa39

This is not the best encryption mechanism, though; for proper encryption, it’s better to use something stronger like DES.

This all sounds good, but what if someone hijacks the traffic between the server and the browser and gets a hold of a properly encrypted email cookie? In this case, the thief would be able to use the same cookie for authentication even without knowing its content. The server would trust the information and let the person into my account. This type of attack is called man-in-the-middle (MITM). To prevent this from happening, we should use HTTPS and inform the browser that the cookie is sensitive and should never be returned to the server without SSL encryption. That’s done by an extra flag in the Set-Cookie header:

HTTP/1.1 303 See Other
Location: www.facebook.com
Set-Cookie: user=me@yegor256.com; Secure

There is yet another type of attack associated with cookie-based authentication, based on a browser’s ability to expose all cookies associated with a web page to JavaScript executed inside it. An attacker may inject some malicious JavaScript code into the page (Don’t ask me how … this will happen only if your entire HTML rendering is done wrong), and this code will gain access to the cookie. Then, the code will send the cookie somewhere else so the attacker can collect it. This type of attack is called cross-site scripting (XSS). To prevent this, there is another flag for the Set-Cookie header, called HttpOnly:

HTTP/1.1 303 See Other
Location: www.facebook.com
Set-Cookie: user=me@yegor256.com; Secure; HttpOnly

The presence of this flag will tell the browser that this particular cookie can be transferred back to the server only through HTTP requests. JavaScript won’t have access to it.

How It’s Done in Takes

Here is how this cookie-based authentication mechanism is designed in the Takes framework. The entire framework consists of takes, which receive requests and produce responses (this article explains the framework in more detail). When the request comes in, we should find the authentication cookie in the Cookie header and translate it to the user credentials. When the response goes out, we should add the Set-Cookie header to it with the encrypted user credentials. That’s it. Just these two steps.

Let’s say we have an account page that is supposed to show the current user’s balance:

final class TkAccount implements Take {
  private final Balances balances;
  @Override
  public Response act(final Request request) {
    final Identity user = // get it from request
    return RsHTML(
      String.format(
        "<html>Your balance is %s</html>",
        this.balances.retrieve(user)
      )
    );
  }
}

Right after the request comes in, we should retrieve the identity of the user, encoded inside an authenticating cookie. To make this mechanism reusable, we have the TkAuth decorator, which wraps an existing take, decodes an incoming cookie, and adds a new TkAuth header to the request with the user’s identification information:

final Codec codec = new CcHex(new CcXOR(new CcPlain()));
final Pass pass = new PsCookie(codec);
new TkAuth(new TkAccount(), pass);

Again, when TkAuth receives a request with an authenticating cookie inside, it asks pass to decode the cookie and return either a valid Identity or Identity.ANONYMOUS.

Then, when the response goes back to the browser, TkAuth asks pass to encode the identity back into a string and adds Set-Cookie to the response.

PsCookie uses an instance of Codec in order to do these backward and forward encoding operations.

When our TkAccount take wants to retrieve a currently authenticated user identity from the request, it can use RqAuth, a utility decorator of Request:

final class TkAccount implements Take {
  @Override
  public Response act(final Request request) {
    final Identity user = new RqAuth(request).identity();
    // other manipulations with the user
  }
}

The RqAuth decorator uses the header, added by PsCookie, in order to authenticate the user and create an Identity object.

How Is It Composable?

This mechanism is indeed very extensible and “composable.” Let’s say we want to skip authentication during integration testing. Here is how:

new TkAuth(
  take, // original application "take"
  new PsChain(
    new PsFake(/* if running integration tests */),
    new PsCookie(
      new CcHex(new CcXOR(new CcPlain()))
    )
  )
);

PsChain implements Pass and attempts to authenticate the user by asking all encapsulated passes, one by one. The first one in the chain is PsFake. Using a single boolean argument in its constructor, it makes a decision whether to return a fake identity or return nothing. With just a single boolean trigger, we can switch off the entire authentication mechanism in the app.

Let’s say you want to authenticate users through Facebook OAuth. Here is how:

new TkAuth(
  take, // original application "take"
  new PsChain(
    new PsByFlag(
      new PsByFlag.Pair(
        PsFacebook.class.getSimpleName(),
        new PsFacebook(
          "... Facebook API key ...",
          "... Facebook API secret ..."
        )
      )
    ),
    new PsCookie(
      new CcHex(new CcXOR(new CcPlain()))
    )
  )
);

When a user clicks on the login link on your site, the browser goes to facebook.com, where his or her identity is verified. Then, Facebook returns a 302 redirection response with a Location header set to the URL we provide in the login link. The link must include something like this: ?PsByFlag=PsFacebook. This will tell PsByFlag that this request authenticates a user.

PsByFlag will iterate through all encapsulated “pairs” and try to find the right one. PsFacebook will be the first and the right one. It will connect to the Facebook API using the provided credentials and will retrieve all possible information about the user.

Here is how we can implement a logout mechanism:

new TkAuth(
  take, // original application "take"
  new PsChain(
    new PsByFlag(
      new PsByFlag.Pair(
        PsFacebook.class.getSimpleName(),
        new PsFacebook(
          "... Facebook API key ...",
          "... Facebook API secret ..."
        )
      ),
      new PsByFlag.Pair(
        PsLogout.class.getSimpleName(),
        new PsLogout()
      )
    ),
    new PsCookie(
      new CcHex(new CcXOR(new CcPlain()))
    )
  )
);

Now, we can add ?PsByFlag=PsLogout to any link on the site and it will log the current user out.

You can see how all this works in a real application by checking out the TkAppAuth class in Rultor.

badge

When you enter your email and password into the Facebook login page, you get into your account. Then, wherever you go in the site, you always see your photo at the top right corner of the page. Facebook remembers you and doesn’t ask for the password again and again. This works thanks to HTTP cookies and is called cookie-based authentication. Even though this mechanism often causes some security problems, it is very popular and simple. Here is how Takes makes it possible in a few lines of code.

First, let’s see how it works. Moreover, let’s see how I believe it should work.

Step one: The user enters an email and password and clicks “submit.” The server receives a POST request with this information inside:

POST / HTTP/1.1
Host: www.facebook.com
Content-Type: application/x-www-form-urlencoded

email=me@yegor256.com&password=itisasecret

The server matches the provided information with its records and decides what to do. If the information is invalid, it returns the same login page, asking you to enter it all again. If the information is valid, the server returns something like this:

HTTP/1.1 303 See Other
Location: www.facebook.com
Set-Cookie: user=me@yegor256.com

Since the response status code is 303, the browser goes to the page specified in the Location header and opens the front page of the site. This is what it sends to the server:

GET / HTTP/1.1
Host: www.facebook.com
Cookie: user=me@yegor256.com

The server gets my email from the Cookie header and understands that it’s me again! No need to ask for the password once more. The server trusts the information from the cookie. That’s it. That’s what cookie-based authentication is all about.

Wait … What About Security?

Right, what about security? If the server trusts any browser request with a user email in the Cookie header, anyone would be able to send my email from another place and get access to my account.

The first step to prevent this is to encrypt the email with a secret encryption key, known only to the server. Nobody except the server itself will be able to encrypt it the same way the server needs to decrypt it. The response would look like this, using an example of encryption by XOR cipher with bamboo as a secret key:

HTTP/1.1 303 See Other
Location: www.facebook.com
Set-Cookie: user=b1ccafd92c568515100f5c4d104671003cfa39

This is not the best encryption mechanism, though; for proper encryption, it’s better to use something stronger like DES.

This all sounds good, but what if someone hijacks the traffic between the server and the browser and gets a hold of a properly encrypted email cookie? In this case, the thief would be able to use the same cookie for authentication even without knowing its content. The server would trust the information and let the person into my account. This type of attack is called man-in-the-middle (MITM). To prevent this from happening, we should use HTTPS and inform the browser that the cookie is sensitive and should never be returned to the server without SSL encryption. That’s done by an extra flag in the Set-Cookie header:

HTTP/1.1 303 See Other
Location: www.facebook.com
Set-Cookie: user=me@yegor256.com; Secure

There is yet another type of attack associated with cookie-based authentication, based on a browser’s ability to expose all cookies associated with a web page to JavaScript executed inside it. An attacker may inject some malicious JavaScript code into the page (Don’t ask me how … this will happen only if your entire HTML rendering is done wrong), and this code will gain access to the cookie. Then, the code will send the cookie somewhere else so the attacker can collect it. This type of attack is called cross-site scripting (XSS). To prevent this, there is another flag for the Set-Cookie header, called HttpOnly:

HTTP/1.1 303 See Other
Location: www.facebook.com
Set-Cookie: user=me@yegor256.com; Secure; HttpOnly

The presence of this flag will tell the browser that this particular cookie can be transferred back to the server only through HTTP requests. JavaScript won’t have access to it.

How It’s Done in Takes

Here is how this cookie-based authentication mechanism is designed in the Takes framework. The entire framework consists of takes, which receive requests and produce responses (this article explains the framework in more detail). When the request comes in, we should find the authentication cookie in the Cookie header and translate it to the user credentials. When the response goes out, we should add the Set-Cookie header to it with the encrypted user credentials. That’s it. Just these two steps.

Let’s say we have an account page that is supposed to show the current user’s balance:

final class TkAccount implements Take {
  private final Balances balances;
  @Override
  public Response act(final Request request) {
    final Identity user = // get it from request
    return RsHTML(
      String.format(
        "<html>Your balance is %s</html>",
        this.balances.retrieve(user)
      )
    );
  }
}

Right after the request comes in, we should retrieve the identity of the user, encoded inside an authenticating cookie. To make this mechanism reusable, we have the TkAuth decorator, which wraps an existing take, decodes an incoming cookie, and adds a new TkAuth header to the request with the user’s identification information:

final Codec codec = new CcHex(new CcXOR(new CcPlain()));
final Pass pass = new PsCookie(codec);
new TkAuth(new TkAccount(), pass);

Again, when TkAuth receives a request with an authenticating cookie inside, it asks pass to decode the cookie and return either a valid Identity or Identity.ANONYMOUS.

Then, when the response goes back to the browser, TkAuth asks pass to encode the identity back into a string and adds Set-Cookie to the response.

PsCookie uses an instance of Codec in order to do these backward and forward encoding operations.

When our TkAccount take wants to retrieve a currently authenticated user identity from the request, it can use RqAuth, a utility decorator of Request:

final class TkAccount implements Take {
  @Override
  public Response act(final Request request) {
    final Identity user = new RqAuth(request).identity();
    // other manipulations with the user
  }
}

The RqAuth decorator uses the header, added by PsCookie, in order to authenticate the user and create an Identity object.

How Is It Composable?

This mechanism is indeed very extensible and “composable.” Let’s say we want to skip authentication during integration testing. Here is how:

new TkAuth(
  take, // original application "take"
  new PsChain(
    new PsFake(/* if running integration tests */),
    new PsCookie(
      new CcHex(new CcXOR(new CcPlain()))
    )
  )
);

PsChain implements Pass and attempts to authenticate the user by asking all encapsulated passes, one by one. The first one in the chain is PsFake. Using a single boolean argument in its constructor, it makes a decision whether to return a fake identity or return nothing. With just a single boolean trigger, we can switch off the entire authentication mechanism in the app.

Let’s say you want to authenticate users through Facebook OAuth. Here is how:

new TkAuth(
  take, // original application "take"
  new PsChain(
    new PsByFlag(
      new PsByFlag.Pair(
        PsFacebook.class.getSimpleName(),
        new PsFacebook(
          "... Facebook API key ...",
          "... Facebook API secret ..."
        )
      )
    ),
    new PsCookie(
      new CcHex(new CcXOR(new CcPlain()))
    )
  )
);

When a user clicks on the login link on your site, the browser goes to facebook.com, where his or her identity is verified. Then, Facebook returns a 302 redirection response with a Location header set to the URL we provide in the login link. The link must include something like this: ?PsByFlag=PsFacebook. This will tell PsByFlag that this request authenticates a user.

PsByFlag will iterate through all encapsulated “pairs” and try to find the right one. PsFacebook will be the first and the right one. It will connect to the Facebook API using the provided credentials and will retrieve all possible information about the user.

Here is how we can implement a logout mechanism:

new TkAuth(
  take, // original application "take"
  new PsChain(
    new PsByFlag(
      new PsByFlag.Pair(
        PsFacebook.class.getSimpleName(),
        new PsFacebook(
          "... Facebook API key ...",
          "... Facebook API secret ..."
        )
      ),
      new PsByFlag.Pair(
        PsLogout.class.getSimpleName(),
        new PsLogout()
      )
    ),
    new PsCookie(
      new CcHex(new CcXOR(new CcPlain()))
    )
  )
);

Now, we can add ?PsByFlag=PsLogout to any link on the site and it will log the current user out.

You can see how all this works in a real application by checking out the TkAppAuth class in Rultor.

© Yegor Bugayenko 2014–2018

Constructors Must Be Code-Free

QR code

Constructors Must Be Code-Free

How much work should be done within a constructor? It seems reasonable to do some computations inside a constructor and then encapsulate results. That way, when the results are required by object methods, we’ll have them ready. Sounds like a good approach? No, it’s not. It’s a bad idea for one reason: It prevents composition of objects and makes them un-extensible.

Kill Bill: Vol. 2 (2004) by Quentin Tarantino
Kill Bill: Vol. 2 (2004) by Quentin Tarantino

Let’s say we’re making an interface that would represent a name of a person:

interface Name {
  String first();
}

Pretty easy, right? Now, let’s try to implement it:

public final class EnglishName implements Name {
  private final String name;
  public EnglishName(final CharSequence text) {
    this.name = text.toString().split(" ", 2)[0];
  }
  @Override
  public String first() {
    return this.name;
  }
}

What’s wrong with this? It’s faster, right? It splits the name into parts only once and encapsulates them. Then, no matter how many times we call the first() method, it will return the same value and won’t need to do the splitting again. However, this is flawed thinking! Let me show you the right way and explain:

public final class EnglishName implements Name {
  private final CharSequence text;
  public EnglishName(final CharSequence txt) {
    this.text = txt;
  }
  @Override
  public String first() {
    return this.text.toString().split("", 2)[0];
  }
}

This is the right design. I can see you smiling, so let me prove my point.

Before I start proving, though, let me ask you to read this article: Composable Decorators vs. Imperative Utility Methods. It explains the difference between a static method and composable decorators. The first snippet above is very close to an imperative utility method, even though it looks like an object. The second example is a true object.

In the first example, we are abusing the new operator and turning it into a static method, which does all calculations for us right here and now. This is what imperative programming is about. In imperative programming, we do all calculations right now and return fully ready results. In declarative programming, we are instead trying to delay calculations for as long as possible.

Let’s try to use our EnglishName class:

final Name name = new EnglishName(
  new NameInPostgreSQL(/*...*/)
);
if (/* something goes wrong */) {
  throw new IllegalStateException(
    String.format(
      "Hi, %s, we can't proceed with your application",
      name.first()
    )
  );
}

In the first line of this snippet, we are just making an instance of an object and labeling it name. We don’t want to go to the database yet and fetch the full name from there, split it into parts, and encapsulate them inside name. We just want to create an instance of an object. Such a parsing behavior would be a side effect for us and, in this case, will slow down the application. As you see, we may only need name.first() if something goes wrong and we need to construct an exception object.

My point is that having any computations done inside a constructor is a bad practice and must be avoided because they are side effects and are not requested by the object owner.

What about performance during the re-use of name, you may ask. If we make an instance of EnglishName and then call name.first() five times, we’ll end up with five calls to the String.split() method.

To solve that, we create another class, a composable decorator, which will help us solve this “re-use” problem:

public final class CachedName implements Name {
  private final Name origin;
  public CachedName(final Name name) {
    this.origin = name;
  }
  @Override
  @Cacheable(forever = true)
  public String first() {
    return this.origin.first();
  }
}

I’m using the Cacheable annotation from jcabi-aspects, but you can use any other caching tools available in Java (or other languages), like Guava Cache:

public final class CachedName implements Name {
  private final Cache<Long, String> cache =
    CacheBuilder.newBuilder().build();
  private final Name origin;
  public CachedName(final Name name) {
    this.origin = name;
  }
  @Override
  public String first() {
    return this.cache.get(
      1L,
      new Callable<String>() {
        @Override
        public String call() {
          return CachedName.this.origin.first();
        }
      }
    );
  }
}

But please don’t make CachedName mutable and lazily loaded—it’s an anti-pattern, which I’ve discussed before in Objects Should Be Immutable.

This is how our code will look now:

final Name name = new CachedName(
  new EnglishName(
    new NameInPostgreSQL(/*...*/)
  )
);

It’s a very primitive example, but I hope you get the idea.

In this design, we’re basically splitting the object into two parts. The first one knows how to get the first name from the English name. The second one knows how to cache the results of this calculation in memory. And now it’s my decision, as a user of these classes, how exactly to use them. I will decide whether I need caching or not. This is what object composition is all about.

Let me reiterate that the only allowed statement inside a constructor is an assignment. If you need to put something else there, start thinking about refactoring—your class definitely needs a redesign.

How much work should be done within a constructor? It seems reasonable to do some computations inside a constructor and then encapsulate results. That way, when the results are required by object methods, we’ll have them ready. Sounds like a good approach? No, it’s not. It’s a bad idea for one reason: It prevents composition of objects and makes them un-extensible.

Kill Bill: Vol. 2 (2004) by Quentin Tarantino
Kill Bill: Vol. 2 (2004) by Quentin Tarantino

Let’s say we’re making an interface that would represent a name of a person:

interface Name {
  String first();
}

Pretty easy, right? Now, let’s try to implement it:

public final class EnglishName implements Name {
  private final String name;
  public EnglishName(final CharSequence text) {
    this.name = text.toString().split(" ", 2)[0];
  }
  @Override
  public String first() {
    return this.name;
  }
}

What’s wrong with this? It’s faster, right? It splits the name into parts only once and encapsulates them. Then, no matter how many times we call the first() method, it will return the same value and won’t need to do the splitting again. However, this is flawed thinking! Let me show you the right way and explain:

public final class EnglishName implements Name {
  private final CharSequence text;
  public EnglishName(final CharSequence txt) {
    this.text = txt;
  }
  @Override
  public String first() {
    return this.text.toString().split("", 2)[0];
  }
}

This is the right design. I can see you smiling, so let me prove my point.

Before I start proving, though, let me ask you to read this article: Composable Decorators vs. Imperative Utility Methods. It explains the difference between a static method and composable decorators. The first snippet above is very close to an imperative utility method, even though it looks like an object. The second example is a true object.

In the first example, we are abusing the new operator and turning it into a static method, which does all calculations for us right here and now. This is what imperative programming is about. In imperative programming, we do all calculations right now and return fully ready results. In declarative programming, we are instead trying to delay calculations for as long as possible.

Let’s try to use our EnglishName class:

final Name name = new EnglishName(
  new NameInPostgreSQL(/*...*/)
);
if (/* something goes wrong */) {
  throw new IllegalStateException(
    String.format(
      "Hi, %s, we can't proceed with your application",
      name.first()
    )
  );
}

In the first line of this snippet, we are just making an instance of an object and labeling it name. We don’t want to go to the database yet and fetch the full name from there, split it into parts, and encapsulate them inside name. We just want to create an instance of an object. Such a parsing behavior would be a side effect for us and, in this case, will slow down the application. As you see, we may only need name.first() if something goes wrong and we need to construct an exception object.

My point is that having any computations done inside a constructor is a bad practice and must be avoided because they are side effects and are not requested by the object owner.

What about performance during the re-use of name, you may ask. If we make an instance of EnglishName and then call name.first() five times, we’ll end up with five calls to the String.split() method.

To solve that, we create another class, a composable decorator, which will help us solve this “re-use” problem:

public final class CachedName implements Name {
  private final Name origin;
  public CachedName(final Name name) {
    this.origin = name;
  }
  @Override
  @Cacheable(forever = true)
  public String first() {
    return this.origin.first();
  }
}

I’m using the Cacheable annotation from jcabi-aspects, but you can use any other caching tools available in Java (or other languages), like Guava Cache:

public final class CachedName implements Name {
  private final Cache<Long, String> cache =
    CacheBuilder.newBuilder().build();
  private final Name origin;
  public CachedName(final Name name) {
    this.origin = name;
  }
  @Override
  public String first() {
    return this.cache.get(
      1L,
      new Callable<String>() {
        @Override
        public String call() {
          return CachedName.this.origin.first();
        }
      }
    );
  }
}

But please don’t make CachedName mutable and lazily loaded—it’s an anti-pattern, which I’ve discussed before in Objects Should Be Immutable.

This is how our code will look now:

final Name name = new CachedName(
  new EnglishName(
    new NameInPostgreSQL(/*...*/)
  )
);

It’s a very primitive example, but I hope you get the idea.

In this design, we’re basically splitting the object into two parts. The first one knows how to get the first name from the English name. The second one knows how to cache the results of this calculation in memory. And now it’s my decision, as a user of these classes, how exactly to use them. I will decide whether I need caching or not. This is what object composition is all about.

Let me reiterate that the only allowed statement inside a constructor is an assignment. If you need to put something else there, start thinking about refactoring—your class definitely needs a redesign.

© Yegor Bugayenko 2014–2018

How to Implement an Iterating Adapter

QR code

How to Implement an Iterating Adapter

  • comments

Iterator is one of the fundamental Java interfaces, introduced in Java 1.2. It is supposed to be very simple; however, in my experience, many Java developers don’t understand how to implement a custom one, which should iterate a stream of data coming from some other source. In other words, it becomes an adapter of another source of data, in the form of an iterator. I hope this example will help.

Let’s say we have an object of this class:

final class Data {
  byte[] read();
}

When we call read(), it returns a new array of bytes that were retrieved from somewhere. If there is nothing to retrieve, the array will be empty. Now, we want to create an adapter that would consume the bytes and let us iterate them:

final class FluentData implements Iterator<Byte> {
  boolean hasNext() { /* ... */ }
  Byte next() { /* ... */ }
  void remove()  { /* ... */ }
}

Here is how it should look (it is not thread-safe!):

final class FluentData implements Iterator<Byte> {
  private final Data data;
  private final Queue<Byte> buffer = new LinkedList<>();
  public FluentData(final Data dat) {
    this.data = dat;
  }
  public boolean hasNext() {
    if (this.buffer.isEmpty()) {
      for (final byte item : this.data.read()) {
        this.buffer.add(item);
      }
    }
    return !this.buffer.isEmpty();
  }
  public Byte next() {
    if (!this.hasNext()) {
      throw new NoSuchElementException("Nothing left");
    }
    return this.buffer.poll();
  }
  public void remove() {
    throw new UnsupportedOperationException("It is read-only");
  }
}

There is no way to make it thread-safe because the iterating process is outside the scope of the iterator. Even if we declare our methods as synchronized, this won’t guarantee that two threads won’t conflict when they both call hasNext() and next(). So don’t bother with it and just document the iterator as not thread-safe, then let its users synchronize one level higher when necessary.

Iterator is one of the fundamental Java interfaces, introduced in Java 1.2. It is supposed to be very simple; however, in my experience, many Java developers don’t understand how to implement a custom one, which should iterate a stream of data coming from some other source. In other words, it becomes an adapter of another source of data, in the form of an iterator. I hope this example will help.

Let’s say we have an object of this class:

final class Data {
  byte[] read();
}

When we call read(), it returns a new array of bytes that were retrieved from somewhere. If there is nothing to retrieve, the array will be empty. Now, we want to create an adapter that would consume the bytes and let us iterate them:

final class FluentData implements Iterator<Byte> {
  boolean hasNext() { /* ... */ }
  Byte next() { /* ... */ }
  void remove()  { /* ... */ }
}

Here is how it should look (it is not thread-safe!):

final class FluentData implements Iterator<Byte> {
  private final Data data;
  private final Queue<Byte> buffer = new LinkedList<>();
  public FluentData(final Data dat) {
    this.data = dat;
  }
  public boolean hasNext() {
    if (this.buffer.isEmpty()) {
      for (final byte item : this.data.read()) {
        this.buffer.add(item);
      }
    }
    return !this.buffer.isEmpty();
  }
  public Byte next() {
    if (!this.hasNext()) {
      throw new NoSuchElementException("Nothing left");
    }
    return this.buffer.poll();
  }
  public void remove() {
    throw new UnsupportedOperationException("It is read-only");
  }
}

There is no way to make it thread-safe because the iterating process is outside the scope of the iterator. Even if we declare our methods as synchronized, this won’t guarantee that two threads won’t conflict when they both call hasNext() and next(). So don’t bother with it and just document the iterator as not thread-safe, then let its users synchronize one level higher when necessary.

© Yegor Bugayenko 2014–2018

JAXB Is Doing It Wrong; Try Xembly

QR code

JAXB Is Doing It Wrong; Try Xembly

  • comments

badge

JAXB is a 10-year-old Java technology that allows us to convert a Java object into an XML document (marshalling) and back (unmarshalling). This technology is based on setters and getters and, in my opinion, violates key principles of object-oriented programming by turning objects into passive data structures. I would recommend you use Xembly instead for marshalling Java objects into XML documents.

This is how JAXB marshalling works. Say you have a Book class that needs to be marshalled into an XML document. You have to create getters and annotate them:

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
@XmlRootElement
public class Book {
  private final String isbn;
  private final String title;
  public Book(final String isbn, final String title) {
    this.isbn = isbn;
    this.title = title;
  }
  @XmlElement
  public String getIsbn() {
    return this.isbn;
  }
  @XmlElement
  public String getTitle() {
    return this.title;
  }
}

Then you create a marshaller and ask it to convert an instance of class Book into XML:

final Book book = new Book("0132350882", "Clean Code");
final JAXBContext context = JAXBContext.newInstance(Book.class);
final Marshaller marshaller = jaxbContext.createMarshaller();
marshaller.marshal(book, System.out);

You should be expecting something like this in the output:

<?xml version="1.0"?>
<book>
  <isbn>0132350882</isbn>
  <title>Clean Code</title>
</book>

So what’s wrong with it? Pretty much the same thing that’s wrong with object-relational mapping, which is explained in ORM Is an Offensive Anti-Pattern. JAXB is treating an object as a bag of data, extracting the data and converting it into XML the way JAXB wants. The object has no control over this process. Therefore an object is not an object anymore but rather a passive bag of data.

An ideal approach would be to redesign our class Book this way:

public class Book {
  private final String isbn;
  private final String title;
  public Book(final String isbn, final String title) {
    this.isbn = isbn;
    this.title = title;
  }
  public String toXML() {
    // create XML document and return
  }
}

However, there are a few problems with this approach. First of all, there’s massive code duplication. Building an XML document is a rather verbose process in Java. If every class had to re-implement it in its toXML() method, we would have a big problem with duplicate code.

The second problem is that we don’t know exactly what type of wrapping our XML document should be delivered in. It may be a String or an InputStream or maybe an instance of org.w3c.dom.Document. Making many toXML() methods in each object would definitely be a disaster.

Xembly provides a solution. As I’ve mentioned before, it is an imperative language for XML constructions and manipulations. Here is how we can implement our Book object with the help of Xembly:

import org.xembly.Directive;
public class Book {
  private final String isbn;
  private final String title;
  public Book(final String isbn, final String title) {
    this.isbn = isbn;
    this.title = title;
  }
  public Iterable<Directive> toXembly() {
    return new Directives()
      .add("book")
      .add("isbn").set(this.isbn).up()
      .add("title").set(this.title).up()
      .up();
  }
}

Now, in order to build an XML document, we should use this code outside the object:

final Book book = new Book("0132350882", "Clean Code");
final String xml = new Xembler(book.toXembly()).xml();

This Xembler class will convert Xembly directives into an XML document.

The beauty of this solution is that the internals of the object are not exposed via getters and the object is fully in charge of the XML marshalling process. In addition, the complexity of these directives may be very high—much higher than the rather cumbersome annotations of JAXB.

Xembly is an open source project, so feel free to submit your questions or corrections to GitHub.

badge

JAXB is a 10-year-old Java technology that allows us to convert a Java object into an XML document (marshalling) and back (unmarshalling). This technology is based on setters and getters and, in my opinion, violates key principles of object-oriented programming by turning objects into passive data structures. I would recommend you use Xembly instead for marshalling Java objects into XML documents.

This is how JAXB marshalling works. Say you have a Book class that needs to be marshalled into an XML document. You have to create getters and annotate them:

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
@XmlRootElement
public class Book {
  private final String isbn;
  private final String title;
  public Book(final String isbn, final String title) {
    this.isbn = isbn;
    this.title = title;
  }
  @XmlElement
  public String getIsbn() {
    return this.isbn;
  }
  @XmlElement
  public String getTitle() {
    return this.title;
  }
}

Then you create a marshaller and ask it to convert an instance of class Book into XML:

final Book book = new Book("0132350882", "Clean Code");
final JAXBContext context = JAXBContext.newInstance(Book.class);
final Marshaller marshaller = jaxbContext.createMarshaller();
marshaller.marshal(book, System.out);

You should be expecting something like this in the output:

<?xml version="1.0"?>
<book>
  <isbn>0132350882</isbn>
  <title>Clean Code</title>
</book>

So what’s wrong with it? Pretty much the same thing that’s wrong with object-relational mapping, which is explained in ORM Is an Offensive Anti-Pattern. JAXB is treating an object as a bag of data, extracting the data and converting it into XML the way JAXB wants. The object has no control over this process. Therefore an object is not an object anymore but rather a passive bag of data.

An ideal approach would be to redesign our class Book this way:

public class Book {
  private final String isbn;
  private final String title;
  public Book(final String isbn, final String title) {
    this.isbn = isbn;
    this.title = title;
  }
  public String toXML() {
    // create XML document and return
  }
}

However, there are a few problems with this approach. First of all, there’s massive code duplication. Building an XML document is a rather verbose process in Java. If every class had to re-implement it in its toXML() method, we would have a big problem with duplicate code.

The second problem is that we don’t know exactly what type of wrapping our XML document should be delivered in. It may be a String or an InputStream or maybe an instance of org.w3c.dom.Document. Making many toXML() methods in each object would definitely be a disaster.

Xembly provides a solution. As I’ve mentioned before, it is an imperative language for XML constructions and manipulations. Here is how we can implement our Book object with the help of Xembly:

import org.xembly.Directive;
public class Book {
  private final String isbn;
  private final String title;
  public Book(final String isbn, final String title) {
    this.isbn = isbn;
    this.title = title;
  }
  public Iterable<Directive> toXembly() {
    return new Directives()
      .add("book")
      .add("isbn").set(this.isbn).up()
      .add("title").set(this.title).up()
      .up();
  }
}

Now, in order to build an XML document, we should use this code outside the object:

final Book book = new Book("0132350882", "Clean Code");
final String xml = new Xembler(book.toXembly()).xml();

This Xembler class will convert Xembly directives into an XML document.

The beauty of this solution is that the internals of the object are not exposed via getters and the object is fully in charge of the XML marshalling process. In addition, the complexity of these directives may be very high—much higher than the rather cumbersome annotations of JAXB.

Xembly is an open source project, so feel free to submit your questions or corrections to GitHub.

© Yegor Bugayenko 2014–2018

Java Web App Architecture In Takes Framework

QR code

Java Web App Architecture In Takes Framework

  • comments

I used to utilize Servlets, JSP, JAX-RS, Spring Framework, Play Framework, JSF with Facelets, and a bit of Spark Framework. All of these solutions, in my humble opinion, are very far from being object-oriented and elegant. They all are full of static methods, un-testable data structures, and dirty hacks. So about a month ago, I decided to create my own Java web framework. I put a few basic principles into its foundation: 1) No NULLs, 2) no public static methods, 3) no mutable classes, and 4) no class casting, reflection, and instanceof operators. These four basic principles should guarantee clean code and transparent architecture. That’s how the Takes framework was born. Let’s see what was created and how it works.

Making of The Godfather (1972) by Francis Ford Coppola
Making of The Godfather (1972) by Francis Ford Coppola

Java Web Architecture in a Nutshell

This is how I understand a web application architecture and its components, in simple terms.

First, to create a web server, we should create a new network socket, that accepts connections on a certain TCP port. Usually it is 80, but I’m going to use 8080 for testing purposes. This is done in Java with the ServerSocket class:

import java.net.ServerSocket;
public class Foo {
  public static void main(final String... args) throws Exception {
    final ServerSocket server = new ServerSocket(8080);
    while (true);
  }
}

That’s enough to start a web server. Now, the socket is ready and listening on port 8080. When someone opens http://localhost:8080 in their browser, the connection will be established and the browser will spin its waiting wheel forever. Compile this snippet and try. We just built a simple web server without the use of any frameworks. We’re not doing anything with incoming connections yet, but we’re not rejecting them either. All of them are being lined up inside that server object. It’s being done in a background thread; that’s why we need to put that while(true) in afterward. Without this endless pause, the app will finish its execution immediately and the server socket will shut down.

The next step is to accept the incoming connections. In Java, that’s done through a blocking call to the accept() method:

final Socket socket = server.accept();

The method is blocking its thread and waiting until a new connection arrives. As soon as that happens, it returns an instance of Socket. In order to accept the next connection, we should call accept() again. So basically, our web server should work like this:

public class Foo {
  public static void main(final String... args) throws Exception {
    final ServerSocket server = new ServerSocket(8080);
    while (true) {
      final Socket socket = server.accept();
      // 1. Read HTTP request from the socket
      // 2. Prepare an HTTP response
      // 3. Send HTTP response to the socket
      // 4. Close the socket
    }
  }
}

It’s an endless cycle that accepts a new connection, understands it, creates a response, returns the response, and accepts a new connection again. HTTP protocol is stateless, which means the server should not remember what happened in any previous connection. All it cares about is the incoming HTTP request in this particular connection.

The HTTP request is coming from the input stream of the socket and looks like a multi-line block of text. This is what you would see if you read an input stream of the socket:

final BufferedReader reader = new BufferedReader(
  new InputStreamReader(socket.getInputStream())
);
while (true) {
  final String line = reader.readLine();
  if (line.isEmpty()) {
    break;
  }
  System.out.println(line);
}

You will see something like this:

GET / HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,ru;q=0.6,uk;q=0.4

The client (the Google Chrome browser, for example) passes this text into the connection established. It connects to port 8080 at localhost, and as soon as the connection is ready, it immediately sends this text into it, then waits for a response.

Our job is to create an HTTP response using the information we get in the request. If our server is very primitive, we can basically ignore all the information in the request and just return “Hello, world!” to all requests (I’m using IOUtils for simplicity):

import java.net.Socket;
import java.net.ServerSocket;
import org.apache.commons.io.IOUtils;
public class Foo {
  public static void main(final String... args) throws Exception {
    final ServerSocket server = new ServerSocket(8080);
    while (true) {
      try (final Socket socket = server.accept()) {
        IOUtils.copy(
          IOUtils.toInputStream("HTTP/1.1 200 OK\r\n\r\nHello, world!"),
          socket.getOutputStream()
        );
      }
    }
  }
}

That’s it. The server is ready. Try to compile and run it. Point your browser to http://localhost:8080, and you will see Hello, world!:

$ javac -cp commons-io.jar Foo.java
$ java -cp commons-io.jar:. Foo &
$ curl http://localhost:8080 -v
* Rebuilt URL to: http://localhost:8080/
* Connected to localhost (::1) port 8080 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.37.1
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 200 OK
* no chunk, no close, no size. Assume close to signal end
<
* Closing connection 0
Hello, world!

That’s all you need to build a web server. Now let’s discuss how to make it object-oriented and composable. Let’s try to see how the Takes framework was built.

Routing/Dispatching

Routing/dispatching is combined with response printing in Takes. All you need to do to create a working web application is to create a single class that implements Take interface:

import org.takes.Request;
import org.takes.Take;
public final class TkFoo implements Take {
  @Override
  public Response route(final Request request) {
    return new RsText("Hello, world!");
  }
}

And now it’s time to start a server:

import org.takes.http.Exit;
import org.takes.http.FtBasic;
public class Foo {
  public static void main(final String... args) throws Exception {
    new FtBasic(new TkFoo(), 8080).start(Exit.NEVER);
  }
}

This FtBasic class does the exact same socket manipulations explained above. It starts a server socket on port 8080 and dispatches all incoming connections through an instance of TkFoo that we are giving to its constructor. It does this dispatching in an endless cycle, checking every second whether it’s time to stop with an instance of Exit. Obviously, Exit.NEVER always responds with, “Don’t stop, please.”

HTTP Request

Now let’s see what’s inside the HTTP request arriving at TkFoo and what we can get out of it. This is how the Request interface is defined in Takes:

public interface Request {
  Iterable<String> head() throws IOException;
  InputStream body() throws IOException;
}

The request is divided into two parts: the head and the body. The head contains all lines that go before the empty line that starts a body, according to HTTP specification in RFC 2616. There are many useful decorators for Request in the framework. For example, RqMethod will help you get the method name from the first line of the header:

final String method = new RqMethod(request).method();

RqHref will help extract the query part and parse it. For example, this is the request:

GET /user?id=123 HTTP/1.1
Host: www.example.com

This code will extract that 123:

final int id = Integer.parseInt(
  new RqHref(request).href().param("id").get(0)
);

RqPrint can get the entire request or its body printed as a String:

final String body = new RqPrint(request).printBody();

The idea here is to keep the Request interface simple and provide this request parsing functionality to its decorators. This approach helps the framework keep classes small and cohesive. Each decorator is very small and solid, doing exactly one thing. All of these decorators are in the org.takes.rq package. As you already probably understand, the Rq prefix stands for Request.

First Real Web App

Let’s create our first real web application, which will do something useful. I would recommend starting with an Entry class, which is required by Java to start an app from the command line:

import org.takes.http.Exit;
import org.takes.http.FtCli;
public final class Entry {
  public static void main(final String... args) throws Exception {
    new FtCli(new TkApp(), args).start(Exit.NEVER);
  }
}

This class contains just a single main() static method that will be called by JVM when the app starts from the command line. As you see, it instantiates FtCli, giving it an instance of class TkApp and command line arguments. We’ll create the TkApp class in a second. FtCli (translates to “front-end with command line interface”) makes an instance of the same FtBasic, wrapping it into a few useful decorators and configuring it according to command line arguments. For example, --port=8080 will be converted into a 8080 port number and passed as a second argument of the FtBasic constructor.

The web application itself is called TkApp and extends TsWrap:

import org.takes.Take;
import org.takes.facets.fork.FkRegex;
import org.takes.facets.fork.TkFork;
import org.takes.tk.TkWrap;
import org.takes.tk.TkClasspath;
final class TkApp extends TkWrap {
  TkApp() {
    super(TkApp.make());
  }
  private static Take make() {
    return new TkFork(
      new FkRegex("/robots.txt", ""),
      new FkRegex("/css/.*", new TkClasspath()),
      new FkRegex("/", new TkIndex())
    );
  }
}

We’ll discuss this TkFork class in a minute.

If you’re using Maven, this is the pom.xml you should start with:

<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
    http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>foo</groupId>
  <artifactId>foo</artifactId>
  <version>1.0-SNAPSHOT</version>
  <dependencies>
    <dependency>
      <groupId>org.takes</groupId>
      <artifactId>takes</artifactId>
      <version>0.9</version> <!-- check the latest in Maven Central -->
    </dependency>
  </dependencies>
  <build>
    <finalName>foo</finalName>
    <plugins>
      <plugin>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>copy-dependencies</goal>
            </goals>
            <configuration>
              <outputDirectory>${project.build.directory}/deps</outputDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Running mvn clean package should build a foo.jar file in target directory and a collection of all JAR dependencies in target/deps. Now you can run the app from the command line:

$ mvn clean package
$ java -Dfile.encoding=UTF-8 \
  -cp ./target/foo.jar:./target/deps/* foo.Entry --port=8080

The application is ready, and you can deploy it to, say, Heroku. Just create a Procfile file in the root of the repository and push the repo to Heroku. This is what Procfile should look like:

web: java -Dfile.encoding=UTF-8 \
  -cp target/foo.jar:target/deps/* \
  foo.Entry --port=${PORT}

TkFork

This TkFork class seems to be one of the core elements of the framework. It helps route an incoming HTTP request to the right take. Its logic is very simple, and there are just a few lines of code inside it. It encapsulates a collection of “forks,” which are instances of the Fork interface:

public interface Fork {
  Iterator<Response> route(Request req) throws IOException;
}

Its only route() method either returns an empty iterator or an iterator with a single Response. TkFork goes through all forks, calling their route() methods until one of them returns a response. Once that happens, TkFork returns this response to the caller, which is FtBasic.

Let’s create a simple fork ourselves now. For example, we want to show the status of the application when the /status URL is requested. Here is the code:

final class TkApp extends TkWrap {
  private static Take make() {
    return new TkFork(
      new Fork() {
        @Override
        public Iterator<Response> route(Request req) {
          final Collection<Response> responses = new ArrayList<>(1);
          if (new RqHref(req).href().path().equals("/status")) {
            responses.add(new TkStatus());
          }
          return responses.iterator();
        }
      }
    );
  }
}

I believe the logic here is clear. We either return an empty iterator or an iterator with an instance of TkStatus inside. If an empty iterator is returned, TkFork will try to find another fork in the collection that actually gets an instance of Response. By the way, if nothing is found and all forks return empty iterators, TkFork will throw a “Page not found” exception.

This exact logic is implemented by an out-of-the-box fork called FkRegex, which attempts to match a request URI path with the regular expression provided:

final class TkApp extends TkWrap {
  private static Take make() {
    return new TkFork(
      new FkRegex("/status", new TkStatus())
    );
  }
}

We can compose a multi-level structure of TkFork classes; for example:

final class TkApp extends TsWrap {
  private static Take make() {
    return new TkFork(
      new FkRegex(
        "/status",
        new TkFork(
          new FkParams("f", "json", new TkStatusJSON()),
          new FkParams("f", "xml", new TkStatusXML())
        )
      )
    );
  }
}

Again, I believe it’s obvious. The instance of FkRegex will ask an encapsulated instance of TkFork to return a response, and it will try to fetch it from one that FkParams encapsulated. If the HTTP query is /status?f=xml, an instance of TkStatusXML will be returned.

HTTP Response

Now let’s discuss the structure of the HTTP response and its object-oriented abstraction, Response. This is how the interface looks:

public interface Response {
  Iterable<String> head() throws IOException;
  InputStream body() throws IOException;
}

Looks very similar to the Request, doesn’t it? Well, it’s identical, mostly because the structure of the HTTP request and response is almost identical. The only difference is the first line.

There is a collection of useful decorators that help in response building. They are composable, which makes them very convenient. For example, if you want to build a response that contains an HTML page, you compose them like this:

final class TkIndex implements Take {
  @Override
  public Response act() {
    return new RsWithStatus(
      new RsWithType(
        new RsWithBody("<html>Hello, world!</html>"),
        "text/html"
      ),
      200
    );
  }
}

In this example, the decorator RsWithBody creates a response with a body but with no headers at all. Then, RsWithType adds the header Content-Type: text/html to it. Then, RsWithStatus makes sure the first line of the response contains HTTP/1.1 200 OK.

You can create your own decorators that can reuse existing ones. Take a look at how it’s done in RsPage from rultor.com.

How About Templates?

Returning simple “Hello, world” pages is not a big problem, as we can see. But what about more complex output like HTML pages, XML documents, JSON data sets, etc? There are a few convenient Response decorators that enable all of that. Let’s start with Velocity, a simple templating engine. Well, it’s not that simple. It’s rather powerful, but I would suggest to use it in simple situations only. Here is how it works:

final class TkIndex implements Take {
  @Override
  public Response act() {
    return new RsVelocity("Hello, ${name}")
      .with("name", "Jeffrey");
  }
}

The RsVelocity constructor accepts a single argument that has to be a Velocity template. Then, you call the with() method, injecting data into the Velocity context. When it’s time to render the HTTP response, RsVelocity will “evaluate” the template against the context configured. Again, I would recommend you use this templating approach only for simple outputs.

For more complex HTML documents, I would recommend you use XML/XSLT in combination with Xembly. I explained this idea in a few previous posts: XML+XSLT in a Browser and RESTful API and a Web Site in the Same URL. It is simple and powerful—Java generates XML output and the XSLT processor transforms it into HTML documents. This is how we separate representation from data. The XSL stylesheet is a “view” and TkIndex is a “controller,” in terms of MVC.

I’ll write a separate article about templating with Xembly and XSL very soon.

In the meantime, we’ll create decorators for JSF/Facelets and JSP rendering in Takes. If you’re interested in helping, please fork the framework and submit your pull requests.

What About Persistence?

Now, a question that comes up is what to do with persistent entities, like databases, in-memory structures, network connections, etc. My suggestion is to initialize them inside the Entry class and pass them as arguments into the TkApp constructor. Then, the TkApp will pass them into the constructors of custom takes.

For example, we have a PostgreSQL database that contains some table data that we need to render. Here is how I would initialize a connection to it in the Entry class (I’m using a BoneCP connection pool):

public final class Entry {
  public static void main(final String... args) throws Exception {
    new FtCli(new TkApp(Entry.postgres()), args).start(Exit.NEVER);
  }
  private static Source postgres() {
    final BoneCPDataSource src = new BoneCPDataSource();
    src.setDriverClass("org.postgresql.Driver");
    src.setJdbcUrl("jdbc:postgresql://localhost/db");
    src.setUser("root");
    src.setPassword("super-secret-password");
    return src;
  }
}

Now, the constructor of TkApp must accept a single argument of type java.sql.Source:

final class TkApp extends TkWrap {
  TkApp(final Source source) {
    super(TkApp.make(source));
  }
  private static Take make(final Source source) {
    return new TkFork(
      new FkRegex("/", new TkIndex(source))
    );
  }
}

Class TkIndex also accepts a single argument of class Source. I believe you know what to do with it inside TkIndex in order to fetch the SQL table data and convert it into HTML. The point here is that the dependency must be injected into the application (instance of class TkApp) at the moment of its instantiation. This is a pure and clean dependency injection mechanism, which is absolutely container-free. Read more about it in Dependency Injection Containers Are Code Polluters.

Unit Testing

Since every class is immutable and all dependencies are injected only through constructors, unit testing is extremely easy. Let’s say we want to test TkStatus, which is supposed to return an HTML response (I’m using JUnit 4 and Hamcrest):

import org.junit.Test;
import org.hamcrest.MatcherAssert;
import org.hamcrest.Matchers;
public final class TkIndexTest {
  @Test
  public void returnsHtmlPage() throws Exception {
    MatcherAssert.assertThat(
      new RsPrint(
        new TkStatus().act(new RqFake())
      ).printBody(),
      Matchers.equalsTo("<html>Hello, world!</html>")
    );
  }
}

Also, we can start the entire application or any individual take in a test HTTP server and test its behavior via a real TCP socket; for example (I’m using jcabi-http to make an HTTP request and check the output):

public final class TkIndexTest {
  @Test
  public void returnsHtmlPage() throws Exception {
    new FtRemote(new TkIndex()).exec(
      new FtRemote.Script() {
        @Override
        public void exec(final URI home) throws IOException {
          new JdkRequest(home)
            .fetch()
            .as(RestResponse.class)
            .assertStatus(HttpURLConnection.HTTP_OK)
            .assertBody(Matchers.containsString("Hello, world!"));
        }
      }
    );
  }
}

FtRemote starts a test web server at a random TCP port and calls the exec() method at the provided instance of FtRemote.Script. The first argument of this method is a URI of the just-started web server homepage.

The architecture of Takes framework is very modular and composable. Any individual take can be tested as a standalone component, absolutely independent from the framework and other takes.

Why the Name?

That’s the question I’ve been hearing rather often. The idea is simple, and it originates from the movie business. When a movie is made, the crew shoots many takes in order to capture the reality and put it on film. Each capture is called a take.

In other words, a take is like a snapshot of the reality.

The same applies to this framework. Each instance of Take represents a reality at one particular moment in time. This reality is then sent to the user in the form of a Response.

PS. There are a few words about authentication: How Cookie-Based Authentication Works in the Takes Framework.

PPS. There are a few real web systems, which you may be interested to take a look at. They all are using Takes Framework and their code is open: rultor.com, jare.io, wring.io.

I used to utilize Servlets, JSP, JAX-RS, Spring Framework, Play Framework, JSF with Facelets, and a bit of Spark Framework. All of these solutions, in my humble opinion, are very far from being object-oriented and elegant. They all are full of static methods, un-testable data structures, and dirty hacks. So about a month ago, I decided to create my own Java web framework. I put a few basic principles into its foundation: 1) No NULLs, 2) no public static methods, 3) no mutable classes, and 4) no class casting, reflection, and instanceof operators. These four basic principles should guarantee clean code and transparent architecture. That’s how the Takes framework was born. Let’s see what was created and how it works.

Making of The Godfather (1972) by Francis Ford Coppola
Making of The Godfather (1972) by Francis Ford Coppola

Java Web Architecture in a Nutshell

This is how I understand a web application architecture and its components, in simple terms.

First, to create a web server, we should create a new network socket, that accepts connections on a certain TCP port. Usually it is 80, but I’m going to use 8080 for testing purposes. This is done in Java with the ServerSocket class:

import java.net.ServerSocket;
public class Foo {
  public static void main(final String... args) throws Exception {
    final ServerSocket server = new ServerSocket(8080);
    while (true);
  }
}

That’s enough to start a web server. Now, the socket is ready and listening on port 8080. When someone opens http://localhost:8080 in their browser, the connection will be established and the browser will spin its waiting wheel forever. Compile this snippet and try. We just built a simple web server without the use of any frameworks. We’re not doing anything with incoming connections yet, but we’re not rejecting them either. All of them are being lined up inside that server object. It’s being done in a background thread; that’s why we need to put that while(true) in afterward. Without this endless pause, the app will finish its execution immediately and the server socket will shut down.

The next step is to accept the incoming connections. In Java, that’s done through a blocking call to the accept() method:

final Socket socket = server.accept();

The method is blocking its thread and waiting until a new connection arrives. As soon as that happens, it returns an instance of Socket. In order to accept the next connection, we should call accept() again. So basically, our web server should work like this:

public class Foo {
  public static void main(final String... args) throws Exception {
    final ServerSocket server = new ServerSocket(8080);
    while (true) {
      final Socket socket = server.accept();
      // 1. Read HTTP request from the socket
      // 2. Prepare an HTTP response
      // 3. Send HTTP response to the socket
      // 4. Close the socket
    }
  }
}

It’s an endless cycle that accepts a new connection, understands it, creates a response, returns the response, and accepts a new connection again. HTTP protocol is stateless, which means the server should not remember what happened in any previous connection. All it cares about is the incoming HTTP request in this particular connection.

The HTTP request is coming from the input stream of the socket and looks like a multi-line block of text. This is what you would see if you read an input stream of the socket:

final BufferedReader reader = new BufferedReader(
  new InputStreamReader(socket.getInputStream())
);
while (true) {
  final String line = reader.readLine();
  if (line.isEmpty()) {
    break;
  }
  System.out.println(line);
}

You will see something like this:

GET / HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,ru;q=0.6,uk;q=0.4

The client (the Google Chrome browser, for example) passes this text into the connection established. It connects to port 8080 at localhost, and as soon as the connection is ready, it immediately sends this text into it, then waits for a response.

Our job is to create an HTTP response using the information we get in the request. If our server is very primitive, we can basically ignore all the information in the request and just return “Hello, world!” to all requests (I’m using IOUtils for simplicity):

import java.net.Socket;
import java.net.ServerSocket;
import org.apache.commons.io.IOUtils;
public class Foo {
  public static void main(final String... args) throws Exception {
    final ServerSocket server = new ServerSocket(8080);
    while (true) {
      try (final Socket socket = server.accept()) {
        IOUtils.copy(
          IOUtils.toInputStream("HTTP/1.1 200 OK\r\n\r\nHello, world!"),
          socket.getOutputStream()
        );
      }
    }
  }
}

That’s it. The server is ready. Try to compile and run it. Point your browser to http://localhost:8080, and you will see Hello, world!:

$ javac -cp commons-io.jar Foo.java
$ java -cp commons-io.jar:. Foo &
$ curl http://localhost:8080 -v
* Rebuilt URL to: http://localhost:8080/
* Connected to localhost (::1) port 8080 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.37.1
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 200 OK
* no chunk, no close, no size. Assume close to signal end
<
* Closing connection 0
Hello, world!

That’s all you need to build a web server. Now let’s discuss how to make it object-oriented and composable. Let’s try to see how the Takes framework was built.

Routing/Dispatching

Routing/dispatching is combined with response printing in Takes. All you need to do to create a working web application is to create a single class that implements Take interface:

import org.takes.Request;
import org.takes.Take;
public final class TkFoo implements Take {
  @Override
  public Response route(final Request request) {
    return new RsText("Hello, world!");
  }
}

And now it’s time to start a server:

import org.takes.http.Exit;
import org.takes.http.FtBasic;
public class Foo {
  public static void main(final String... args) throws Exception {
    new FtBasic(new TkFoo(), 8080).start(Exit.NEVER);
  }
}

This FtBasic class does the exact same socket manipulations explained above. It starts a server socket on port 8080 and dispatches all incoming connections through an instance of TkFoo that we are giving to its constructor. It does this dispatching in an endless cycle, checking every second whether it’s time to stop with an instance of Exit. Obviously, Exit.NEVER always responds with, “Don’t stop, please.”

HTTP Request

Now let’s see what’s inside the HTTP request arriving at TkFoo and what we can get out of it. This is how the Request interface is defined in Takes:

public interface Request {
  Iterable<String> head() throws IOException;
  InputStream body() throws IOException;
}

The request is divided into two parts: the head and the body. The head contains all lines that go before the empty line that starts a body, according to HTTP specification in RFC 2616. There are many useful decorators for Request in the framework. For example, RqMethod will help you get the method name from the first line of the header:

final String method = new RqMethod(request).method();

RqHref will help extract the query part and parse it. For example, this is the request:

GET /user?id=123 HTTP/1.1
Host: www.example.com

This code will extract that 123:

final int id = Integer.parseInt(
  new RqHref(request).href().param("id").get(0)
);

RqPrint can get the entire request or its body printed as a String:

final String body = new RqPrint(request).printBody();

The idea here is to keep the Request interface simple and provide this request parsing functionality to its decorators. This approach helps the framework keep classes small and cohesive. Each decorator is very small and solid, doing exactly one thing. All of these decorators are in the org.takes.rq package. As you already probably understand, the Rq prefix stands for Request.

First Real Web App

Let’s create our first real web application, which will do something useful. I would recommend starting with an Entry class, which is required by Java to start an app from the command line:

import org.takes.http.Exit;
import org.takes.http.FtCli;
public final class Entry {
  public static void main(final String... args) throws Exception {
    new FtCli(new TkApp(), args).start(Exit.NEVER);
  }
}

This class contains just a single main() static method that will be called by JVM when the app starts from the command line. As you see, it instantiates FtCli, giving it an instance of class TkApp and command line arguments. We’ll create the TkApp class in a second. FtCli (translates to “front-end with command line interface”) makes an instance of the same FtBasic, wrapping it into a few useful decorators and configuring it according to command line arguments. For example, --port=8080 will be converted into a 8080 port number and passed as a second argument of the FtBasic constructor.

The web application itself is called TkApp and extends TsWrap:

import org.takes.Take;
import org.takes.facets.fork.FkRegex;
import org.takes.facets.fork.TkFork;
import org.takes.tk.TkWrap;
import org.takes.tk.TkClasspath;
final class TkApp extends TkWrap {
  TkApp() {
    super(TkApp.make());
  }
  private static Take make() {
    return new TkFork(
      new FkRegex("/robots.txt", ""),
      new FkRegex("/css/.*", new TkClasspath()),
      new FkRegex("/", new TkIndex())
    );
  }
}

We’ll discuss this TkFork class in a minute.

If you’re using Maven, this is the pom.xml you should start with:

<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
    http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>foo</groupId>
  <artifactId>foo</artifactId>
  <version>1.0-SNAPSHOT</version>
  <dependencies>
    <dependency>
      <groupId>org.takes</groupId>
      <artifactId>takes</artifactId>
      <version>0.9</version> <!-- check the latest in Maven Central -->
    </dependency>
  </dependencies>
  <build>
    <finalName>foo</finalName>
    <plugins>
      <plugin>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>copy-dependencies</goal>
            </goals>
            <configuration>
              <outputDirectory>${project.build.directory}/deps</outputDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Running mvn clean package should build a foo.jar file in target directory and a collection of all JAR dependencies in target/deps. Now you can run the app from the command line:

$ mvn clean package
$ java -Dfile.encoding=UTF-8 \
  -cp ./target/foo.jar:./target/deps/* foo.Entry --port=8080

The application is ready, and you can deploy it to, say, Heroku. Just create a Procfile file in the root of the repository and push the repo to Heroku. This is what Procfile should look like:

web: java -Dfile.encoding=UTF-8 \
  -cp target/foo.jar:target/deps/* \
  foo.Entry --port=${PORT}

TkFork

This TkFork class seems to be one of the core elements of the framework. It helps route an incoming HTTP request to the right take. Its logic is very simple, and there are just a few lines of code inside it. It encapsulates a collection of “forks,” which are instances of the Fork interface:

public interface Fork {
  Iterator<Response> route(Request req) throws IOException;
}

Its only route() method either returns an empty iterator or an iterator with a single Response. TkFork goes through all forks, calling their route() methods until one of them returns a response. Once that happens, TkFork returns this response to the caller, which is FtBasic.

Let’s create a simple fork ourselves now. For example, we want to show the status of the application when the /status URL is requested. Here is the code:

final class TkApp extends TkWrap {
  private static Take make() {
    return new TkFork(
      new Fork() {
        @Override
        public Iterator<Response> route(Request req) {
          final Collection<Response> responses = new ArrayList<>(1);
          if (new RqHref(req).href().path().equals("/status")) {
            responses.add(new TkStatus());
          }
          return responses.iterator();
        }
      }
    );
  }
}

I believe the logic here is clear. We either return an empty iterator or an iterator with an instance of TkStatus inside. If an empty iterator is returned, TkFork will try to find another fork in the collection that actually gets an instance of Response. By the way, if nothing is found and all forks return empty iterators, TkFork will throw a “Page not found” exception.

This exact logic is implemented by an out-of-the-box fork called FkRegex, which attempts to match a request URI path with the regular expression provided:

final class TkApp extends TkWrap {
  private static Take make() {
    return new TkFork(
      new FkRegex("/status", new TkStatus())
    );
  }
}

We can compose a multi-level structure of TkFork classes; for example:

final class TkApp extends TsWrap {
  private static Take make() {
    return new TkFork(
      new FkRegex(
        "/status",
        new TkFork(
          new FkParams("f", "json", new TkStatusJSON()),
          new FkParams("f", "xml", new TkStatusXML())
        )
      )
    );
  }
}

Again, I believe it’s obvious. The instance of FkRegex will ask an encapsulated instance of TkFork to return a response, and it will try to fetch it from one that FkParams encapsulated. If the HTTP query is /status?f=xml, an instance of TkStatusXML will be returned.

HTTP Response

Now let’s discuss the structure of the HTTP response and its object-oriented abstraction, Response. This is how the interface looks:

public interface Response {
  Iterable<String> head() throws IOException;
  InputStream body() throws IOException;
}

Looks very similar to the Request, doesn’t it? Well, it’s identical, mostly because the structure of the HTTP request and response is almost identical. The only difference is the first line.

There is a collection of useful decorators that help in response building. They are composable, which makes them very convenient. For example, if you want to build a response that contains an HTML page, you compose them like this:

final class TkIndex implements Take {
  @Override
  public Response act() {
    return new RsWithStatus(
      new RsWithType(
        new RsWithBody("<html>Hello, world!</html>"),
        "text/html"
      ),
      200
    );
  }
}

In this example, the decorator RsWithBody creates a response with a body but with no headers at all. Then, RsWithType adds the header Content-Type: text/html to it. Then, RsWithStatus makes sure the first line of the response contains HTTP/1.1 200 OK.

You can create your own decorators that can reuse existing ones. Take a look at how it’s done in RsPage from rultor.com.

How About Templates?

Returning simple “Hello, world” pages is not a big problem, as we can see. But what about more complex output like HTML pages, XML documents, JSON data sets, etc? There are a few convenient Response decorators that enable all of that. Let’s start with Velocity, a simple templating engine. Well, it’s not that simple. It’s rather powerful, but I would suggest to use it in simple situations only. Here is how it works:

final class TkIndex implements Take {
  @Override
  public Response act() {
    return new RsVelocity("Hello, ${name}")
      .with("name", "Jeffrey");
  }
}

The RsVelocity constructor accepts a single argument that has to be a Velocity template. Then, you call the with() method, injecting data into the Velocity context. When it’s time to render the HTTP response, RsVelocity will “evaluate” the template against the context configured. Again, I would recommend you use this templating approach only for simple outputs.

For more complex HTML documents, I would recommend you use XML/XSLT in combination with Xembly. I explained this idea in a few previous posts: XML+XSLT in a Browser and RESTful API and a Web Site in the Same URL. It is simple and powerful—Java generates XML output and the XSLT processor transforms it into HTML documents. This is how we separate representation from data. The XSL stylesheet is a “view” and TkIndex is a “controller,” in terms of MVC.

I’ll write a separate article about templating with Xembly and XSL very soon.

In the meantime, we’ll create decorators for JSF/Facelets and JSP rendering in Takes. If you’re interested in helping, please fork the framework and submit your pull requests.

What About Persistence?

Now, a question that comes up is what to do with persistent entities, like databases, in-memory structures, network connections, etc. My suggestion is to initialize them inside the Entry class and pass them as arguments into the TkApp constructor. Then, the TkApp will pass them into the constructors of custom takes.

For example, we have a PostgreSQL database that contains some table data that we need to render. Here is how I would initialize a connection to it in the Entry class (I’m using a BoneCP connection pool):

public final class Entry {
  public static void main(final String... args) throws Exception {
    new FtCli(new TkApp(Entry.postgres()), args).start(Exit.NEVER);
  }
  private static Source postgres() {
    final BoneCPDataSource src = new BoneCPDataSource();
    src.setDriverClass("org.postgresql.Driver");
    src.setJdbcUrl("jdbc:postgresql://localhost/db");
    src.setUser("root");
    src.setPassword("super-secret-password");
    return src;
  }
}

Now, the constructor of TkApp must accept a single argument of type java.sql.Source:

final class TkApp extends TkWrap {
  TkApp(final Source source) {
    super(TkApp.make(source));
  }
  private static Take make(final Source source) {
    return new TkFork(
      new FkRegex("/", new TkIndex(source))
    );
  }
}

Class TkIndex also accepts a single argument of class Source. I believe you know what to do with it inside TkIndex in order to fetch the SQL table data and convert it into HTML. The point here is that the dependency must be injected into the application (instance of class TkApp) at the moment of its instantiation. This is a pure and clean dependency injection mechanism, which is absolutely container-free. Read more about it in Dependency Injection Containers Are Code Polluters.

Unit Testing

Since every class is immutable and all dependencies are injected only through constructors, unit testing is extremely easy. Let’s say we want to test TkStatus, which is supposed to return an HTML response (I’m using JUnit 4 and Hamcrest):

import org.junit.Test;
import org.hamcrest.MatcherAssert;
import org.hamcrest.Matchers;
public final class TkIndexTest {
  @Test
  public void returnsHtmlPage() throws Exception {
    MatcherAssert.assertThat(
      new RsPrint(
        new TkStatus().act(new RqFake())
      ).printBody(),
      Matchers.equalsTo("<html>Hello, world!</html>")
    );
  }
}

Also, we can start the entire application or any individual take in a test HTTP server and test its behavior via a real TCP socket; for example (I’m using jcabi-http to make an HTTP request and check the output):

public final class TkIndexTest {
  @Test
  public void returnsHtmlPage() throws Exception {
    new FtRemote(new TkIndex()).exec(
      new FtRemote.Script() {
        @Override
        public void exec(final URI home) throws IOException {
          new JdkRequest(home)
            .fetch()
            .as(RestResponse.class)
            .assertStatus(HttpURLConnection.HTTP_OK)
            .assertBody(Matchers.containsString("Hello, world!"));
        }
      }
    );
  }
}

FtRemote starts a test web server at a random TCP port and calls the exec() method at the provided instance of FtRemote.Script. The first argument of this method is a URI of the just-started web server homepage.

The architecture of Takes framework is very modular and composable. Any individual take can be tested as a standalone component, absolutely independent from the framework and other takes.

Why the Name?

That’s the question I’ve been hearing rather often. The idea is simple, and it originates from the movie business. When a movie is made, the crew shoots many takes in order to capture the reality and put it on film. Each capture is called a take.

In other words, a take is like a snapshot of the reality.

The same applies to this framework. Each instance of Take represents a reality at one particular moment in time. This reality is then sent to the user in the form of a Response.

PS. There are a few words about authentication: How Cookie-Based Authentication Works in the Takes Framework.

PPS. There are a few real web systems, which you may be interested to take a look at. They all are using Takes Framework and their code is open: rultor.com, jare.io, wring.io.

© Yegor Bugayenko 2014–2018

Don't Repeat Yourself in Maven POMs; Use Jcabi-Parent

QR code

Don't Repeat Yourself in Maven POMs; Use Jcabi-Parent

  • comments

badge

Maven is a build automation tool mostly for Java projects. It’s a great tool, but it has one important drawback that has motivated the creation of similar tools, like Gradle and SBT. That weakness is its verbosity of configuration. Maven gets all project build parameters from pom.xml, an XML file that can get very long. I’ve seen POM files of 3,000-plus lines. Taking into account 1) recent DSL buzz and 2) fear of XML, it’s only logical that many people don’t like Maven because of its pom.xml verbosity.

But even if you’re an XML fan who enjoys its strictness and elegance (like myself), you won’t like the necessity to repeat yourself in pom.xml for every project. If you’re working on multiple projects, code duplication will be enormous. An average Java web app uses a few dozen standard Maven plugins and almost the same number of pretty common dependencies, like JUnit, Apache Commons, Log4J, Mockito, etc. All of them have their versions and configurations, which have to be specified if you want to keep the project stable and avoid Maven warnings. Thus, once a new version of a plugin is released, you have to go through all pom.xml files in the projects you’re working on and update it there. You obviously understand what code duplication means. It’s a disaster. However, there is a solution.

jcabi-parent is a very simple Maven dependency with nothing inside it except a large pom.xml with multiple pre-configured dependencies, profiles, and plugins. All you need to do in order to reuse them all in your project is define com.jcabi:parent as your parent POM:

<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
    http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>com.jcabi</groupId>
    <artifactId>parent</artifactId>
    <!-- check the latest version at http://parent.jcabi.com -->
    <version>0.32.1</version>
  </parent>
  [...]
</project>

That’s all you need. Now you can remove most of your custom configurations from pom.xml and rely on defaults provided by jcabi-parent. Its pom.xml is rather large and properly configured. Multiple projects depend on it, so you can be confident that you’re using the best possible configuration of all standard plugins.

Here are a few examples of pom.xml from projects that are using jcabi-parent (you can see how compact they are): Xembly ReXSL jcabi-http Qulice

badge

Maven is a build automation tool mostly for Java projects. It’s a great tool, but it has one important drawback that has motivated the creation of similar tools, like Gradle and SBT. That weakness is its verbosity of configuration. Maven gets all project build parameters from pom.xml, an XML file that can get very long. I’ve seen POM files of 3,000-plus lines. Taking into account 1) recent DSL buzz and 2) fear of XML, it’s only logical that many people don’t like Maven because of its pom.xml verbosity.

But even if you’re an XML fan who enjoys its strictness and elegance (like myself), you won’t like the necessity to repeat yourself in pom.xml for every project. If you’re working on multiple projects, code duplication will be enormous. An average Java web app uses a few dozen standard Maven plugins and almost the same number of pretty common dependencies, like JUnit, Apache Commons, Log4J, Mockito, etc. All of them have their versions and configurations, which have to be specified if you want to keep the project stable and avoid Maven warnings. Thus, once a new version of a plugin is released, you have to go through all pom.xml files in the projects you’re working on and update it there. You obviously understand what code duplication means. It’s a disaster. However, there is a solution.

jcabi-parent is a very simple Maven dependency with nothing inside it except a large pom.xml with multiple pre-configured dependencies, profiles, and plugins. All you need to do in order to reuse them all in your project is define com.jcabi:parent as your parent POM:

<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
    http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>com.jcabi</groupId>
    <artifactId>parent</artifactId>
    <!-- check the latest version at http://parent.jcabi.com -->
    <version>0.32.1</version>
  </parent>
  [...]
</project>

That’s all you need. Now you can remove most of your custom configurations from pom.xml and rely on defaults provided by jcabi-parent. Its pom.xml is rather large and properly configured. Multiple projects depend on it, so you can be confident that you’re using the best possible configuration of all standard plugins.

Here are a few examples of pom.xml from projects that are using jcabi-parent (you can see how compact they are): Xembly ReXSL jcabi-http Qulice

© Yegor Bugayenko 2014–2018

XSL Transformation in Java: An Easy Way

QR code

XSL Transformation in Java: An Easy Way

  • comments

badge

XSL transformation (XSLT) is a powerful mechanism for converting one XML document into another. However, in Java, XML manipulations are rather verbose and complex. Even for a simple XSL transformation, you have to write a few dozen lines of code—and maybe even more than that if proper exception handling and logging is needed. jcabi-xml is a small open source library that makes life much easier by enabling XML parsing and XPath traversing with a few simple methods. Let’s see how this library helps in XSL transformations.

First, take a look at a practical example—rultor.com—a hosted DevOps assistant that automates release, merge, and deploy operations. Rultor keeps each conversation session with an end user (a.k.a. “talk”) in a DynamoDB record. There are multiple situations to handle in each talk; that’s why using multiple columns of a record is not really feasible. Instead, we’re keeping only a few parameters of each talk in record columns (like ID and name) and putting all the rest in a single XML column.

This is approximately how our DynamoDB table looks:

+----+---------------+--------------------------------------+
| id | name          | xml                                  |
+----+---------------+--------------------------------------+
| 12 | jcabi-xml#54  | <?xml version='1.0'?>                |
|    |               | <talk public="true">                 |
|    |               |   <request id="e5f4b3">...</request> |
|    |               | </talk>                              |
+----+---------------+--------------------------------------+
| 13 | jcabi-email#2 | <?xml version='1.0'?>                |
|    |               | <talk public="true">                 |
|    |               |   <daemon id="f787fe">...</daemon>   |
|    |               | </talk>                              |
+----+---------------+--------------------------------------+

Once a user posts @rultor status into a GitHub ticket, Rultor has to answer with a full status report about the current talk. In order to create such a text answer (a regular user would not appreciate an XML response), we have to fetch that xml column from the necessary DynamoDB record and convert it to plain English text.

Here is how we’re doing that with the help of jcabi-xml and its class, XSLDocument.

final String xml = // comes from DynamoDB
final XSL xsl = new XSLDocument(
  this.getClass().getResourceAsStream("status.xsl")
);
final String text = xsl.applyTo(xml);

That’s it. Now let’s see what’s there in that status.xsl file (this is just a skeleton of it; the full version is here):

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  version="2.0">
  <xsl:output method="text"/>
  <xsl:template match="/talk">
    <xsl:text>Hi, here is your status report:</xsl:text>
    ...
  </xsl:template>
  <xsl:template match="node()|@*">
    <xsl:copy>
      <xsl:apply-templates select="node()|@*"/>
    </xsl:copy>
  </xsl:template>
</xsl:stylesheet>

It is good practice to create XSL documents only once per application run. We have a static utility method XSLDocument.make() for this:

final class Foo {
  private static final XSL STYLESHEET = XSLDocument.make(
    Foo.class.getResourceAsStream("stylesheet.xsl")
  );
  public XML style(final XML xml) {
    return Foo.STYLESHEET.transform(xml);
  }
}

Pay attention to the fact we’re using XSLT 2.0. Built-in Java implementation of XSLT doesn’t support version 2.0, and in order to make it work, we’re using these two Maven Saxon dependencies:

<dependency>
  <groupId>net.sourceforge.saxon</groupId>
  <artifactId>saxon</artifactId>
  <version>9.1.0.8</version>
  <scope>runtime</scope>
</dependency>
<dependency>
  <groupId>net.sourceforge.saxon</groupId>
  <artifactId>saxon</artifactId>
  <version>9.1.0.8</version>
  <classifier>xpath</classifier>
  <scope>runtime</scope>
</dependency>

All you need to do to start using jcabi-xml for XSL transformations is add this dependency to your pom.xml:

<dependency>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-xml</artifactId>
</dependency>

If you have any problems or suggestions, don’t hesitate to submit an issue to the GitHub issue tracker.

badge

XSL transformation (XSLT) is a powerful mechanism for converting one XML document into another. However, in Java, XML manipulations are rather verbose and complex. Even for a simple XSL transformation, you have to write a few dozen lines of code—and maybe even more than that if proper exception handling and logging is needed. jcabi-xml is a small open source library that makes life much easier by enabling XML parsing and XPath traversing with a few simple methods. Let’s see how this library helps in XSL transformations.

First, take a look at a practical example—rultor.com—a hosted DevOps assistant that automates release, merge, and deploy operations. Rultor keeps each conversation session with an end user (a.k.a. “talk”) in a DynamoDB record. There are multiple situations to handle in each talk; that’s why using multiple columns of a record is not really feasible. Instead, we’re keeping only a few parameters of each talk in record columns (like ID and name) and putting all the rest in a single XML column.

This is approximately how our DynamoDB table looks:

+----+---------------+--------------------------------------+
| id | name          | xml                                  |
+----+---------------+--------------------------------------+
| 12 | jcabi-xml#54  | <?xml version='1.0'?>                |
|    |               | <talk public="true">                 |
|    |               |   <request id="e5f4b3">...</request> |
|    |               | </talk>                              |
+----+---------------+--------------------------------------+
| 13 | jcabi-email#2 | <?xml version='1.0'?>                |
|    |               | <talk public="true">                 |
|    |               |   <daemon id="f787fe">...</daemon>   |
|    |               | </talk>                              |
+----+---------------+--------------------------------------+

Once a user posts @rultor status into a GitHub ticket, Rultor has to answer with a full status report about the current talk. In order to create such a text answer (a regular user would not appreciate an XML response), we have to fetch that xml column from the necessary DynamoDB record and convert it to plain English text.

Here is how we’re doing that with the help of jcabi-xml and its class, XSLDocument.

final String xml = // comes from DynamoDB
final XSL xsl = new XSLDocument(
  this.getClass().getResourceAsStream("status.xsl")
);
final String text = xsl.applyTo(xml);

That’s it. Now let’s see what’s there in that status.xsl file (this is just a skeleton of it; the full version is here):

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  version="2.0">
  <xsl:output method="text"/>
  <xsl:template match="/talk">
    <xsl:text>Hi, here is your status report:</xsl:text>
    ...
  </xsl:template>
  <xsl:template match="node()|@*">
    <xsl:copy>
      <xsl:apply-templates select="node()|@*"/>
    </xsl:copy>
  </xsl:template>
</xsl:stylesheet>

It is good practice to create XSL documents only once per application run. We have a static utility method XSLDocument.make() for this:

final class Foo {
  private static final XSL STYLESHEET = XSLDocument.make(
    Foo.class.getResourceAsStream("stylesheet.xsl")
  );
  public XML style(final XML xml) {
    return Foo.STYLESHEET.transform(xml);
  }
}

Pay attention to the fact we’re using XSLT 2.0. Built-in Java implementation of XSLT doesn’t support version 2.0, and in order to make it work, we’re using these two Maven Saxon dependencies:

<dependency>
  <groupId>net.sourceforge.saxon</groupId>
  <artifactId>saxon</artifactId>
  <version>9.1.0.8</version>
  <scope>runtime</scope>
</dependency>
<dependency>
  <groupId>net.sourceforge.saxon</groupId>
  <artifactId>saxon</artifactId>
  <version>9.1.0.8</version>
  <classifier>xpath</classifier>
  <scope>runtime</scope>
</dependency>

All you need to do to start using jcabi-xml for XSL transformations is add this dependency to your pom.xml:

<dependency>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-xml</artifactId>
</dependency>

If you have any problems or suggestions, don’t hesitate to submit an issue to the GitHub issue tracker.

© Yegor Bugayenko 2014–2018

If. Then. Throw. Else. WTF?

QR code

If. Then. Throw. Else. WTF?

  • comments

This is the code I could never understand:

if (x < 0) {
  throw new Exception("X can't be negative");
} else {
  System.out.println("X is positive or zero");
}

I have been trying to find a proper metaphor to explain its incorrectness. Today I finally found it.

If-then-else is a forking mechanism of procedural programming. The CPU either goes to the left and then does something or goes to the right and does something else. Imagine yourself driving a car and seeing this sign:

The figure

It looks logical, doesn’t it? You can go in the left lane if you’re not driving a truck. Otherwise you should go in the right lane. Both lanes meet up in a while. No matter which one you choose, you will end up on the same road. This is what this code block does:

if (x < 0) {
  System.out.println("X is negative");
} else {
  System.out.println("X is positive or zero");
}

Now, try to imagine this sign:

The figure

It looks very strange to me, and you will never see this sign anywhere simply because a dead end means an end, a full stop, a finish. What is the point of drawing a lane after the dead end sign? There is no point.

This is how a proper sign would look:

The figure

This is how a proper code block would look:

if (x < 0) {
  throw new Exception("X can't be negative");
}
System.out.println("X is positive or zero");

The same is true for loops. This is wrong:

for (int x : numbers) {
  if (x < 0) {
    continue;
  } else {
    System.out.println("found positive number");
  }
}

While this is right:

for (int x : numbers) {
  if (x < 0) {
    continue;
  }
  System.out.println("found positive number");
}

There is no road after the dead end! If you draw it, your code looks like this very funny snippet I found a few years ago reviewing sources written by some very well-paid developer in one very serious company:

if (x < 0) {
  throw new Exception("X is negative");
  System.exit(1);
}

Don’t do this.

This is the code I could never understand:

if (x < 0) {
  throw new Exception("X can't be negative");
} else {
  System.out.println("X is positive or zero");
}

I have been trying to find a proper metaphor to explain its incorrectness. Today I finally found it.

If-then-else is a forking mechanism of procedural programming. The CPU either goes to the left and then does something or goes to the right and does something else. Imagine yourself driving a car and seeing this sign:

The figure

It looks logical, doesn’t it? You can go in the left lane if you’re not driving a truck. Otherwise you should go in the right lane. Both lanes meet up in a while. No matter which one you choose, you will end up on the same road. This is what this code block does:

if (x < 0) {
  System.out.println("X is negative");
} else {
  System.out.println("X is positive or zero");
}

Now, try to imagine this sign:

The figure

It looks very strange to me, and you will never see this sign anywhere simply because a dead end means an end, a full stop, a finish. What is the point of drawing a lane after the dead end sign? There is no point.

This is how a proper sign would look:

The figure

This is how a proper code block would look:

if (x < 0) {
  throw new Exception("X can't be negative");
}
System.out.println("X is positive or zero");

The same is true for loops. This is wrong:

for (int x : numbers) {
  if (x < 0) {
    continue;
  } else {
    System.out.println("found positive number");
  }
}

While this is right:

for (int x : numbers) {
  if (x < 0) {
    continue;
  }
  System.out.println("found positive number");
}

There is no road after the dead end! If you draw it, your code looks like this very funny snippet I found a few years ago reviewing sources written by some very well-paid developer in one very serious company:

if (x < 0) {
  throw new Exception("X is negative");
  System.exit(1);
}

Don’t do this.

© Yegor Bugayenko 2014–2018

How Immutability Helps

QR code

How Immutability Helps

  • comments

In a few recent posts, including Getters/Setters. Evil. Period. Objects Should Be Immutable, and Dependency Injection Containers are Code Polluters, I universally labeled all mutable objects with “setters” (object methods starting with set) evil. My argumentation was based mostly on metaphors and abstract examples. Apparently, this wasn’t convincing enough for many of you—I received a few requests asking to provide more specific and practical examples.

Thus, in order to illustrate my strongly negative attitude to “mutability via setters,” I took an existing commons-email Java library from Apache and re-designed it my way, without setters and with “object thinking” in mind. I released my library as part of the jcabi family—jcabi-email. Let’s see what benefits we get from a “pure” object-oriented and immutable approach, without getters.

Here is how your code will look, if you send an email using commons-email:

Email email = new SimpleEmail();
email.setHostName("smtp.googlemail.com");
email.setSmtpPort(465);
email.setAuthenticator(new DefaultAuthenticator("user", "pwd"));
email.setFrom("yegor256@gmail.com", "Yegor Bugayenko");
email.addTo("dude@jcabi.com");
email.setSubject("how are you?");
email.setMsg("Dude, how are you?");
email.send();

Here is how you do the same with jcabi-email:

Postman postman = new Postman.Default(
  new SMTP("smtp.googlemail.com", 465, "user", "pwd")
);
Envelope envelope = new Envelope.MIME(
  new Array<Stamp>(
    new StSender("Yegor Bugayenko <yegor256@gmail.com>"),
    new StRecipient("dude@jcabi.com"),
    new StSubject("how are you?")
  ),
  new Array<Enclosure>(
    new EnPlain("Dude, how are you?")
  )
);
postman.send(envelope);

I think the difference is obvious.

In the first example, you’re dealing with a monster class that can do everything for you, including sending your MIME message via SMTP, creating the message, configuring its parameters, adding MIME parts to it, etc. The Email class from commons-email is really a huge class—33 private properties, over a hundred methods, about two thousands lines of code. First, you configure the class through a bunch of setters and then you ask it to send() an email for you.

In the second example, we have seven objects instantiated via seven new calls. Postman is responsible for packaging a MIME message; SMTP is responsible for sending it via SMTP; stamps (StSender, StRecipient, and StSubject) are responsible for configuring the MIME message before delivery; enclosure EnPlain is responsible for creating a MIME part for the message we’re going to send. We construct these seven objects, encapsulating one into another, and then we ask the postman to send() the envelope for us.

What’s Wrong With a Mutable Email?

From a user perspective, there is almost nothing wrong. Email is a powerful class with multiple controls—just hit the right one and the job gets done. However, from a developer perspective Email class is a nightmare. Mostly because the class is very big and difficult to maintain.

Because the class is so big, every time you want to extend it by introducing a new method, you’re facing the fact that you’re making the class even worse—longer, less cohesive, less readable, less maintainable, etc. You have a feeling that you’re digging into something dirty and that there is no hope to make it cleaner, ever. I’m sure, you’re familiar with this feeling—most legacy applications look that way. They have huge multi-line “classes” (in reality, COBOL programs written in Java) that were inherited from a few generations of programmers before you. When you start, you’re full of energy, but after a few minutes of scrolling such a “class” you say—“screw it, it’s almost Saturday.”

Because the class is so big, there is no data hiding or encapsulation any more—33 variables are accessible by over 100 methods. What is hidden? This Email.java file in reality is a big, procedural 2000-line script, called a “class” by mistake. Nothing is hidden, once you cross the border of the class by calling one of its methods. After that, you have full access to all the data you may need. Why is this bad? Well, why do we need encapsulation in the first place? In order to protect one programmer from another, aka defensive programming. While I’m busy changing the subject of the MIME message, I want to be sure that I’m not interfered with by some other method’s activity, that is changing a sender and touching my subject by mistake. Encapsulation helps us narrow down the scope of the problem, while this Email class is doing exactly the opposite.

Because the class is so big, its unit testing is even more complicated than the class itself. Why? Because of multiple inter-dependencies between its methods and properties. In order to test setCharset() you have to prepare the entire object by calling a few other methods, then you have to call send() to make sure the message being sent actually uses the encoding you specified. Thus, in order to test a one-line method setCharset() you run the entire integration testing scenario of sending a full MIME message through SMTP. Obviously, if something gets changed in one of the methods, almost every test method will be affected. In other words, tests are very fragile, unreliable and over-complicated.

I can go on and on with this “because the class is so big,” but I think it is obvious that a small, cohesive class is always better than a big one. It is obvious to me, to you, and to any object-oriented programmer. But why is it not so obvious to the developers of Apache Commons Email? I don’t think they are stupid or un-educated. What is it then?

How and Why Did It Happen?

This is how it always happens. You start to design a class as something cohesive, solid, and small. Your intentions are very positive. Very soon you realize that there is something else that this class has to do. Then, something else. Then, even more.

The best way to make your class more and more powerful is by adding setters that inject configuration parameters into the class so that it can process them inside, isn’t it?

This is the root cause of the problem! The root cause is our ability to insert data into mutable objects via configuration methods, also known as “setters.” When an object is mutable and allows us to add setters whenever we want, we will do it without limits.

Let me put it this way—_mutable classes tend to grow in size and lose cohesiveness_.

If commons-email authors made this Email class immutable in the beginning, they wouldn’t have been able to add so many methods into it and encapsulate so many properties. They wouldn’t be able to turn it into a monster. Why? Because an immutable object only accepts a state through a constructor. Can you imagine a 33-argument constructor? Of course, not.

When you make your class immutable in the first place, you are forced to keep it cohesive, small, solid and robust. Because you can’t encapsulate too much and you can’t modify what’s encapsulated. Just two or three arguments of a constructor and you’re done.

How Did I Design An Immutable Email?

badge

When I was designing jcabi-email I started with a small and simple class: Postman. Well, it is an interface, since I never make interface-less classes. So, Postman is… a post man. He is delivering messages to other people. First, I created a default version of it (I omit the ctor, for the sake of brevity):

import javax.mail.Message;
@Immutable
class Postman.Default implements Postman {
  private final String host;
  private final int port;
  private final String user;
  private final String password;
  @Override
  void send(Message msg) {
    // create SMTP session
    // create transport
    // transport.connect(this.host, this.port, etc.)
    // transport.send(msg)
    // transport.close();
  }
}

Good start, it works. What now? Well, the Message is difficult to construct. It is a complex class from JDK that requires some manipulations before it can become a nice HTML email. So I created an envelope, which will build this complex object for me (pay attention, both Postman and Envelope are immutable and annotated with @Immutable from jcabi-aspects):

@Immutable
interface Envelope {
  Message unwrap();
}

I also refactor the Postman to accept an envelope, not a message:

@Immutable
interface Postman {
  void send(Envelope env);
}

So far, so good. Now let’s try to create a simple implementation of Envelope:

@Immutable
class MIME implements Envelope {
  @Override
  public Message unwrap() {
    return new MimeMessage(
      Session.getDefaultInstance(new Properties())
    );
  }
}

It works, but it does nothing useful yet. It only creates an absolutely empty MIME message and returns it. How about adding a subject to it and both To: and From: addresses (pay attention, MIME class is also immutable):

@Immutable
class Envelope.MIME implements Envelope {
  private final String subject;
  private final String from;
  private final Array<String> to;
  public MIME(String subj, String sender, Iterable<String> rcpts) {
    this.subject = subj;
    this.from = sender;
    this.to = new Array<String>(rcpts);
  }
  @Override
  public Message unwrap() {
    Message msg = new MimeMessage(
      Session.getDefaultInstance(new Properties())
    );
    msg.setSubject(this.subject);
    msg.setFrom(new InternetAddress(this.from));
    for (String email : this.to) {
      msg.setRecipient(
        Message.RecipientType.TO,
        new InternetAddress(email)
      );
    }
    return msg;
  }
}

Looks correct and it works. But it is still too primitive. How about CC: and BCC:? What about email text? How about PDF enclosures? What if I want to specify the encoding of the message? What about Reply-To?

Can I add all these parameters to the constructor? Remember, the class is immutable and I can’t introduce the setReplyTo() method. I have to pass the replyTo argument into its constructor. It’s impossible, because the constructor will have too many arguments, and nobody will be able to use it.

So, what do I do?

Well, I started to think: how can we break the concept of an “envelope” into smaller concepts—and this what I invented. Like a real-life envelope, my MIME object will have stamps. Stamps will be responsible for configuring an object Message (again, Stamp is immutable, as well as all its implementers):

@Immutable
interface Stamp {
  void attach(Message message);
}

Now, I can simplify my MIME class to the following:

@Immutable
class Envelope.MIME implements Envelope {
  private final Array<Stamp> stamps;
  public MIME(Iterable<Stamp> stmps) {
    this.stamps = new Array<Stamp>(stmps);
  }
  @Override
  public Message unwrap() {
    Message msg = new MimeMessage(
      Session.getDefaultInstance(new Properties())
    );
    for (Stamp stamp : this.stamps) {
      stamp.attach(msg);
    }
    return msg;
  }
}

Now, I will create stamps for the subject, for To:, for From:, for CC:, for BCC:, etc. As many stamps as I like. The class MIME will stay the same—small, cohesive, readable, solid, etc.

What is important here is why I made the decision to refactor while the class was relatively small. Indeed, I started to worry about these stamp classes when my MIME class was just 25 lines in size.

That is exactly the point of this article—_immutability forces you to design small and cohesive objects_.

Without immutability, I would have gone the same direction as commons-email. My MIME class would grow in size and sooner or later would become as big as Email from commons-email. The only thing that stopped me was the necessity to refactor it, because I wasn’t able to pass all arguments through a constructor.

Without immutability, I wouldn’t have had that motivator and I would have done what Apache developers did with commons-email—bloat the class and turn it into an unmaintainable monster.

That’s jcabi-email. I hope this example was illustrative enough and that you will start writing cleaner code with immutable objects.

In a few recent posts, including Getters/Setters. Evil. Period. Objects Should Be Immutable, and Dependency Injection Containers are Code Polluters, I universally labeled all mutable objects with “setters” (object methods starting with set) evil. My argumentation was based mostly on metaphors and abstract examples. Apparently, this wasn’t convincing enough for many of you—I received a few requests asking to provide more specific and practical examples.

Thus, in order to illustrate my strongly negative attitude to “mutability via setters,” I took an existing commons-email Java library from Apache and re-designed it my way, without setters and with “object thinking” in mind. I released my library as part of the jcabi family—jcabi-email. Let’s see what benefits we get from a “pure” object-oriented and immutable approach, without getters.

Here is how your code will look, if you send an email using commons-email:

Email email = new SimpleEmail();
email.setHostName("smtp.googlemail.com");
email.setSmtpPort(465);
email.setAuthenticator(new DefaultAuthenticator("user", "pwd"));
email.setFrom("yegor256@gmail.com", "Yegor Bugayenko");
email.addTo("dude@jcabi.com");
email.setSubject("how are you?");
email.setMsg("Dude, how are you?");
email.send();

Here is how you do the same with jcabi-email:

Postman postman = new Postman.Default(
  new SMTP("smtp.googlemail.com", 465, "user", "pwd")
);
Envelope envelope = new Envelope.MIME(
  new Array<Stamp>(
    new StSender("Yegor Bugayenko <yegor256@gmail.com>"),
    new StRecipient("dude@jcabi.com"),
    new StSubject("how are you?")
  ),
  new Array<Enclosure>(
    new EnPlain("Dude, how are you?")
  )
);
postman.send(envelope);

I think the difference is obvious.

In the first example, you’re dealing with a monster class that can do everything for you, including sending your MIME message via SMTP, creating the message, configuring its parameters, adding MIME parts to it, etc. The Email class from commons-email is really a huge class—33 private properties, over a hundred methods, about two thousands lines of code. First, you configure the class through a bunch of setters and then you ask it to send() an email for you.

In the second example, we have seven objects instantiated via seven new calls. Postman is responsible for packaging a MIME message; SMTP is responsible for sending it via SMTP; stamps (StSender, StRecipient, and StSubject) are responsible for configuring the MIME message before delivery; enclosure EnPlain is responsible for creating a MIME part for the message we’re going to send. We construct these seven objects, encapsulating one into another, and then we ask the postman to send() the envelope for us.

What’s Wrong With a Mutable Email?

From a user perspective, there is almost nothing wrong. Email is a powerful class with multiple controls—just hit the right one and the job gets done. However, from a developer perspective Email class is a nightmare. Mostly because the class is very big and difficult to maintain.

Because the class is so big, every time you want to extend it by introducing a new method, you’re facing the fact that you’re making the class even worse—longer, less cohesive, less readable, less maintainable, etc. You have a feeling that you’re digging into something dirty and that there is no hope to make it cleaner, ever. I’m sure, you’re familiar with this feeling—most legacy applications look that way. They have huge multi-line “classes” (in reality, COBOL programs written in Java) that were inherited from a few generations of programmers before you. When you start, you’re full of energy, but after a few minutes of scrolling such a “class” you say—“screw it, it’s almost Saturday.”

Because the class is so big, there is no data hiding or encapsulation any more—33 variables are accessible by over 100 methods. What is hidden? This Email.java file in reality is a big, procedural 2000-line script, called a “class” by mistake. Nothing is hidden, once you cross the border of the class by calling one of its methods. After that, you have full access to all the data you may need. Why is this bad? Well, why do we need encapsulation in the first place? In order to protect one programmer from another, aka defensive programming. While I’m busy changing the subject of the MIME message, I want to be sure that I’m not interfered with by some other method’s activity, that is changing a sender and touching my subject by mistake. Encapsulation helps us narrow down the scope of the problem, while this Email class is doing exactly the opposite.

Because the class is so big, its unit testing is even more complicated than the class itself. Why? Because of multiple inter-dependencies between its methods and properties. In order to test setCharset() you have to prepare the entire object by calling a few other methods, then you have to call send() to make sure the message being sent actually uses the encoding you specified. Thus, in order to test a one-line method setCharset() you run the entire integration testing scenario of sending a full MIME message through SMTP. Obviously, if something gets changed in one of the methods, almost every test method will be affected. In other words, tests are very fragile, unreliable and over-complicated.

I can go on and on with this “because the class is so big,” but I think it is obvious that a small, cohesive class is always better than a big one. It is obvious to me, to you, and to any object-oriented programmer. But why is it not so obvious to the developers of Apache Commons Email? I don’t think they are stupid or un-educated. What is it then?

How and Why Did It Happen?

This is how it always happens. You start to design a class as something cohesive, solid, and small. Your intentions are very positive. Very soon you realize that there is something else that this class has to do. Then, something else. Then, even more.

The best way to make your class more and more powerful is by adding setters that inject configuration parameters into the class so that it can process them inside, isn’t it?

This is the root cause of the problem! The root cause is our ability to insert data into mutable objects via configuration methods, also known as “setters.” When an object is mutable and allows us to add setters whenever we want, we will do it without limits.

Let me put it this way—_mutable classes tend to grow in size and lose cohesiveness_.

If commons-email authors made this Email class immutable in the beginning, they wouldn’t have been able to add so many methods into it and encapsulate so many properties. They wouldn’t be able to turn it into a monster. Why? Because an immutable object only accepts a state through a constructor. Can you imagine a 33-argument constructor? Of course, not.

When you make your class immutable in the first place, you are forced to keep it cohesive, small, solid and robust. Because you can’t encapsulate too much and you can’t modify what’s encapsulated. Just two or three arguments of a constructor and you’re done.

How Did I Design An Immutable Email?

badge

When I was designing jcabi-email I started with a small and simple class: Postman. Well, it is an interface, since I never make interface-less classes. So, Postman is… a post man. He is delivering messages to other people. First, I created a default version of it (I omit the ctor, for the sake of brevity):

import javax.mail.Message;
@Immutable
class Postman.Default implements Postman {
  private final String host;
  private final int port;
  private final String user;
  private final String password;
  @Override
  void send(Message msg) {
    // create SMTP session
    // create transport
    // transport.connect(this.host, this.port, etc.)
    // transport.send(msg)
    // transport.close();
  }
}

Good start, it works. What now? Well, the Message is difficult to construct. It is a complex class from JDK that requires some manipulations before it can become a nice HTML email. So I created an envelope, which will build this complex object for me (pay attention, both Postman and Envelope are immutable and annotated with @Immutable from jcabi-aspects):

@Immutable
interface Envelope {
  Message unwrap();
}

I also refactor the Postman to accept an envelope, not a message:

@Immutable
interface Postman {
  void send(Envelope env);
}

So far, so good. Now let’s try to create a simple implementation of Envelope:

@Immutable
class MIME implements Envelope {
  @Override
  public Message unwrap() {
    return new MimeMessage(
      Session.getDefaultInstance(new Properties())
    );
  }
}

It works, but it does nothing useful yet. It only creates an absolutely empty MIME message and returns it. How about adding a subject to it and both To: and From: addresses (pay attention, MIME class is also immutable):

@Immutable
class Envelope.MIME implements Envelope {
  private final String subject;
  private final String from;
  private final Array<String> to;
  public MIME(String subj, String sender, Iterable<String> rcpts) {
    this.subject = subj;
    this.from = sender;
    this.to = new Array<String>(rcpts);
  }
  @Override
  public Message unwrap() {
    Message msg = new MimeMessage(
      Session.getDefaultInstance(new Properties())
    );
    msg.setSubject(this.subject);
    msg.setFrom(new InternetAddress(this.from));
    for (String email : this.to) {
      msg.setRecipient(
        Message.RecipientType.TO,
        new InternetAddress(email)
      );
    }
    return msg;
  }
}

Looks correct and it works. But it is still too primitive. How about CC: and BCC:? What about email text? How about PDF enclosures? What if I want to specify the encoding of the message? What about Reply-To?

Can I add all these parameters to the constructor? Remember, the class is immutable and I can’t introduce the setReplyTo() method. I have to pass the replyTo argument into its constructor. It’s impossible, because the constructor will have too many arguments, and nobody will be able to use it.

So, what do I do?

Well, I started to think: how can we break the concept of an “envelope” into smaller concepts—and this what I invented. Like a real-life envelope, my MIME object will have stamps. Stamps will be responsible for configuring an object Message (again, Stamp is immutable, as well as all its implementers):

@Immutable
interface Stamp {
  void attach(Message message);
}

Now, I can simplify my MIME class to the following:

@Immutable
class Envelope.MIME implements Envelope {
  private final Array<Stamp> stamps;
  public MIME(Iterable<Stamp> stmps) {
    this.stamps = new Array<Stamp>(stmps);
  }
  @Override
  public Message unwrap() {
    Message msg = new MimeMessage(
      Session.getDefaultInstance(new Properties())
    );
    for (Stamp stamp : this.stamps) {
      stamp.attach(msg);
    }
    return msg;
  }
}

Now, I will create stamps for the subject, for To:, for From:, for CC:, for BCC:, etc. As many stamps as I like. The class MIME will stay the same—small, cohesive, readable, solid, etc.

What is important here is why I made the decision to refactor while the class was relatively small. Indeed, I started to worry about these stamp classes when my MIME class was just 25 lines in size.

That is exactly the point of this article—_immutability forces you to design small and cohesive objects_.

Without immutability, I would have gone the same direction as commons-email. My MIME class would grow in size and sooner or later would become as big as Email from commons-email. The only thing that stopped me was the necessity to refactor it, because I wasn’t able to pass all arguments through a constructor.

Without immutability, I wouldn’t have had that motivator and I would have done what Apache developers did with commons-email—bloat the class and turn it into an unmaintainable monster.

That’s jcabi-email. I hope this example was illustrative enough and that you will start writing cleaner code with immutable objects.

© Yegor Bugayenko 2014–2018

Paired Brackets

QR code

Paired Brackets

  • comments

Here is a notation rule I’m using in Java code: a bracket should either start/end a line or be paired on the same line.

The notation applies universally to any programming language (incl. Java, Ruby, Python, C++, PHP, etc.) where brackets are used for method/function calls.

Here is how your code will look, if you follow this “Paired Brackets” notation:

new Foo( // ends the line
  Math.max(10, 40), // open/close at the same line
  String.format(
    "hello, %s",
    new Name(
      Arrays.asList(
        "Jeff",
        "Lebowski"
      )
    )
  ) // starts the line
);

Obviously, the line with a closing bracket should start at the same indentation level as the line with its opening pair.

This is how your IDE will render the code if you follow this notation (IntelliJ IDEA):

The figure

Sublime Text will also appreciate it:

The figure

As you see, those light vertical lines at the left side of the code help you to navigate, if you follow the notation.

Those multiple closing brackets may look strange to you at the beginning—but give yourself some time and you will get used to them :)

Fluent

This is how I would recommend formatting fluent method calls (this is Java in NetBeans):

The figure

Arrays

Here is how you format an array in “Paired Brackets” notation (this is Ruby in RubyMine):

The figure

As you see, the same principle applies to square and curled brackets.

JSON

The same principle is applicable to JSON formatting. This is a small JSON document in Coda 2:

The figure

JavaScript

JavaScript should also follow the same principle. This is how your .js code would look in Atom:

The figure

Python

Finally, here is Python in PyCharm:

The figure

Here is a notation rule I’m using in Java code: a bracket should either start/end a line or be paired on the same line.

The notation applies universally to any programming language (incl. Java, Ruby, Python, C++, PHP, etc.) where brackets are used for method/function calls.

Here is how your code will look, if you follow this “Paired Brackets” notation:

new Foo( // ends the line
  Math.max(10, 40), // open/close at the same line
  String.format(
    "hello, %s",
    new Name(
      Arrays.asList(
        "Jeff",
        "Lebowski"
      )
    )
  ) // starts the line
);

Obviously, the line with a closing bracket should start at the same indentation level as the line with its opening pair.

This is how your IDE will render the code if you follow this notation (IntelliJ IDEA):

The figure

Sublime Text will also appreciate it:

The figure

As you see, those light vertical lines at the left side of the code help you to navigate, if you follow the notation.

Those multiple closing brackets may look strange to you at the beginning—but give yourself some time and you will get used to them :)

Fluent

This is how I would recommend formatting fluent method calls (this is Java in NetBeans):

The figure

Arrays

Here is how you format an array in “Paired Brackets” notation (this is Ruby in RubyMine):

The figure

As you see, the same principle applies to square and curled brackets.

JSON

The same principle is applicable to JSON formatting. This is a small JSON document in Coda 2:

The figure

JavaScript

JavaScript should also follow the same principle. This is how your .js code would look in Atom:

The figure

Python

Finally, here is Python in PyCharm:

The figure

© Yegor Bugayenko 2014–2018

Built-in Fake Objects

QR code

Built-in Fake Objects

  • comments

While mock objects are perfect instruments for unit testing, mocking through mock frameworks may turn your unit tests into an unmaintainable mess. Thanks to them we often hear that “mocking is bad” and “mocking is evil.”

The root cause of this complexity is that our objects are too big. They have many methods and these methods return other objects, which also have methods. When we pass a mock version of such an object as a parameter, we should make sure that all of its methods return valid objects.

This leads to inevitable complexity, which turns unit tests to waste almost impossible to maintain.

Object Hierarchy

Take the Region interface from jcabi-dynamo as an example (this snippet and all others in this article are simplified, for the sake of brevity):

public interface Region {
  Table table(String name);
}

Its table() method returns an instance of the Table interface, which has its own methods:

public interface Table {
  Frame frame();
  Item put(Attributes attrs);
  Region region();
}

Interface Frame, returned by the frame() method, also has its own methods. And so on. In order to create a properly mocked instance of interface Region, one would normally create a dozen other mock objects. With Mockito it will look like this:

public void testMe() {
  // many more lines here...
  Frame frame = Mockito.mock(Frame.class);
  Mockito.doReturn(...).when(frame).iterator();
  Table table = Mockito.mock(Table.class);
  Mockito.doReturn(frame).when(table).frame();
  Region region = Mockito.mock(Region.class);
  Mockito.doReturn(table).when(region).table(Mockito.anyString());
}

And all of this is just a scaffolding before the actual testing.

Sample Use Case

Let’s say, you’re developing a project that uses jcabi-dynamo for managing data in DynamoDB. Your class may look similar to this:

public class Employee {
  private final String name;
  private final Region region;
  public Employee(String empl, Region dynamo) {
    this.name = empl;
    this.region = dynamo;
  }
  public Integer salary() {
    return Integer.parseInt(
      this.region
        .table("employees")
        .frame()
        .where("name", this.name)
        .iterator()
        .next()
        .get("salary")
        .getN()
    );
  }
}

You can imagine how difficult it will be to unit test this class, using Mockito, for example. First, we have to mock the Region interface. Then, we have to mock a Table interface and make sure it is returned by the table() method. Then, we have to mock a Frame interface, etc.

The unit test will be much longer than the class itself. Besides that, its real purpose, which is to test the retrieval of an employee’s salary, will not be obvious to the reader.

Moreover, when we need to test a similar method of a similar class, we will need to restart this mocking from scratch. Again, multiple lines of code, which will look very similar to what we have already written.

Fake Classes

The solution is to create fake classes and ship them together with real classes. This is what jcabi-dynamo is doing. Just look at its JavaDoc. There is a package called com.jcabi.dynamo.mock that contains only fake classes, suitable only for unit testing.

Even though their sole purpose is to optimize unit testing, we ship them together with production code, in the same JAR package.

This is what a test will look like, when a fake class MkRegion is used:

public class EmployeeTest {
  public void canFetchSalaryFromDynamoDb() {
    Region region = new MkRegion(
      new H2Data().with(
        "employees", new String[] {"name"},
        new String[] {"salary"}
      )
    );
    region.table("employees").put(
      new Attributes()
        .with("name", "Jeff")
        .with("salary", new AttributeValue().withN(50000))
    );
    Employee emp = new Employee("Jeff", region);
    assertThat(emp.salary(), equalTo(50000));
  }
}

This test looks obvious to me. First, we create a fake DynamoDB region, which works on top of H2Data storage (in-memory H2 database). The storage will be ready for a single employees table with a hash key name and a single salary attribute.

Then, we put a record into the table, with a hash Jeff and a salary 50000.

Finally, we create an instance of class Employee and check how it fetches the salary from DynamoDB.

I’m currently doing the same thing in almost every open source library I’m working with. I’m creating a collection of fake classes, that simplify testing inside the library and for its users.

BTW, a great article on the same subject: tl;dw: Stop mocking, start testing by Ned Batchelder.

PS. Check this out, on a very similar subject: Mocking of HTTP Server in Java.

While mock objects are perfect instruments for unit testing, mocking through mock frameworks may turn your unit tests into an unmaintainable mess. Thanks to them we often hear that “mocking is bad” and “mocking is evil.”

The root cause of this complexity is that our objects are too big. They have many methods and these methods return other objects, which also have methods. When we pass a mock version of such an object as a parameter, we should make sure that all of its methods return valid objects.

This leads to inevitable complexity, which turns unit tests to waste almost impossible to maintain.

Object Hierarchy

Take the Region interface from jcabi-dynamo as an example (this snippet and all others in this article are simplified, for the sake of brevity):

public interface Region {
  Table table(String name);
}

Its table() method returns an instance of the Table interface, which has its own methods:

public interface Table {
  Frame frame();
  Item put(Attributes attrs);
  Region region();
}

Interface Frame, returned by the frame() method, also has its own methods. And so on. In order to create a properly mocked instance of interface Region, one would normally create a dozen other mock objects. With Mockito it will look like this:

public void testMe() {
  // many more lines here...
  Frame frame = Mockito.mock(Frame.class);
  Mockito.doReturn(...).when(frame).iterator();
  Table table = Mockito.mock(Table.class);
  Mockito.doReturn(frame).when(table).frame();
  Region region = Mockito.mock(Region.class);
  Mockito.doReturn(table).when(region).table(Mockito.anyString());
}

And all of this is just a scaffolding before the actual testing.

Sample Use Case

Let’s say, you’re developing a project that uses jcabi-dynamo for managing data in DynamoDB. Your class may look similar to this:

public class Employee {
  private final String name;
  private final Region region;
  public Employee(String empl, Region dynamo) {
    this.name = empl;
    this.region = dynamo;
  }
  public Integer salary() {
    return Integer.parseInt(
      this.region
        .table("employees")
        .frame()
        .where("name", this.name)
        .iterator()
        .next()
        .get("salary")
        .getN()
    );
  }
}

You can imagine how difficult it will be to unit test this class, using Mockito, for example. First, we have to mock the Region interface. Then, we have to mock a Table interface and make sure it is returned by the table() method. Then, we have to mock a Frame interface, etc.

The unit test will be much longer than the class itself. Besides that, its real purpose, which is to test the retrieval of an employee’s salary, will not be obvious to the reader.

Moreover, when we need to test a similar method of a similar class, we will need to restart this mocking from scratch. Again, multiple lines of code, which will look very similar to what we have already written.

Fake Classes

The solution is to create fake classes and ship them together with real classes. This is what jcabi-dynamo is doing. Just look at its JavaDoc. There is a package called com.jcabi.dynamo.mock that contains only fake classes, suitable only for unit testing.

Even though their sole purpose is to optimize unit testing, we ship them together with production code, in the same JAR package.

This is what a test will look like, when a fake class MkRegion is used:

public class EmployeeTest {
  public void canFetchSalaryFromDynamoDb() {
    Region region = new MkRegion(
      new H2Data().with(
        "employees", new String[] {"name"},
        new String[] {"salary"}
      )
    );
    region.table("employees").put(
      new Attributes()
        .with("name", "Jeff")
        .with("salary", new AttributeValue().withN(50000))
    );
    Employee emp = new Employee("Jeff", region);
    assertThat(emp.salary(), equalTo(50000));
  }
}

This test looks obvious to me. First, we create a fake DynamoDB region, which works on top of H2Data storage (in-memory H2 database). The storage will be ready for a single employees table with a hash key name and a single salary attribute.

Then, we put a record into the table, with a hash Jeff and a salary 50000.

Finally, we create an instance of class Employee and check how it fetches the salary from DynamoDB.

I’m currently doing the same thing in almost every open source library I’m working with. I’m creating a collection of fake classes, that simplify testing inside the library and for its users.

BTW, a great article on the same subject: tl;dw: Stop mocking, start testing by Ned Batchelder.

PS. Check this out, on a very similar subject: Mocking of HTTP Server in Java.

© Yegor Bugayenko 2014–2018

Deploying to Heroku, in One Click

QR code

Deploying to Heroku, in One Click

  • comments

There were a few articles already about our usage of Rultor for automating continuous delivery cycles of Java and Ruby projects, including RubyGems, CloudBees and Maven Central.

This one describes how Heroku deployment can be automated. When I need to deploy a new version of an Aintshy web application, all I do is create one message in a GitHub ticket. I just say @rultor release 0.1.4 and version 0.1.4 gets deployed to Heroku. See GitHub ticket #5.

You can do the same, with the help of Rultor.com, a free hosted DevOps assistant.

Create Heroku Project

Create a new project at Heroku.com.

Then install their command line tool-belt.

Authenticate at Heroku

You should authenticate your public SSH key at Heroku, using their command line tool-belt. The process is explained here, but it is not much of a process. You just run heroku login and enter your login credentials. As a result, you will get your existing key (located at ~/.ssh/id_rsa.pub) authenticated by Heroku.

If you didn’t have the key before, it will be created automatically.

Encrypt SSH Key

Now, encrypt id_rsa and id_rsa.pub (they are in the ~/.ssh directory) with a rultor remote:

$ gem install rultor
$ rultor encrypt -p me/test id_rsa
$ rultor encrypt -p me/test id_rsa.pub

Instead of me/test use the name of your GitHub project.

You will get two new files id_rsa.asc and id_rsa.pub.asc. Add them to the root directory of your project, commit and push. These files contain your secret information, but only the Rultor server can decrypt them.

Create Rultor Configuration

Create a .rultor.yml file in the root directory of your project (reference page explains this format in detail):

decrypt:
  id_rsa: "repo/id_rsa.asc"
  id_rsa.pub: "repo/id_rsa.pub.asc"
release:
  script: |
    mvn versions:set "-DnewVersion=${tag}"
    git commit -am "${tag}"
    mvn clean install -Pqulice --errors
    git remote add heroku git@heroku.com:aintshy.git
    mkdir ~/.ssh
    mv ../id_rsa ../id_rsa.pub ~/.ssh
    chmod -R 600 ~/.ssh/*
    echo -e \
      "Host *\n  StrictHostKeyChecking no\n  UserKnownHostsFile=/dev/null" \
      > ~/.ssh/config
    git push -f heroku $(git symbolic-ref --short HEAD):master

You can compare your file with live Rultor configuration of aintshy/hub.

Run It!

badge

Now it’s time to see how it all works. Create a new ticket in the GitHub issue tracker, and post something like this into it (read more about Rultor commands):

@rultor release, tag is `0.1`

You will get a response in a few seconds. The rest will be done by Rultor.

Enjoy :)

BTW, if something doesn’t work as I’ve explained, don’t hesitate to submit a ticket to the Rultor issue tracker. I will try to help you.

PS. I would also recommend to versionalize artifacts through MANIFEST.MF and use jcabi-manifests to read them later.

There were a few articles already about our usage of Rultor for automating continuous delivery cycles of Java and Ruby projects, including RubyGems, CloudBees and Maven Central.

This one describes how Heroku deployment can be automated. When I need to deploy a new version of an Aintshy web application, all I do is create one message in a GitHub ticket. I just say @rultor release 0.1.4 and version 0.1.4 gets deployed to Heroku. See GitHub ticket #5.

You can do the same, with the help of Rultor.com, a free hosted DevOps assistant.

Create Heroku Project

Create a new project at Heroku.com.

Then install their command line tool-belt.

Authenticate at Heroku

You should authenticate your public SSH key at Heroku, using their command line tool-belt. The process is explained here, but it is not much of a process. You just run heroku login and enter your login credentials. As a result, you will get your existing key (located at ~/.ssh/id_rsa.pub) authenticated by Heroku.

If you didn’t have the key before, it will be created automatically.

Encrypt SSH Key

Now, encrypt id_rsa and id_rsa.pub (they are in the ~/.ssh directory) with a rultor remote:

$ gem install rultor
$ rultor encrypt -p me/test id_rsa
$ rultor encrypt -p me/test id_rsa.pub

Instead of me/test use the name of your GitHub project.

You will get two new files id_rsa.asc and id_rsa.pub.asc. Add them to the root directory of your project, commit and push. These files contain your secret information, but only the Rultor server can decrypt them.

Create Rultor Configuration

Create a .rultor.yml file in the root directory of your project (reference page explains this format in detail):

decrypt:
  id_rsa: "repo/id_rsa.asc"
  id_rsa.pub: "repo/id_rsa.pub.asc"
release:
  script: |
    mvn versions:set "-DnewVersion=${tag}"
    git commit -am "${tag}"
    mvn clean install -Pqulice --errors
    git remote add heroku git@heroku.com:aintshy.git
    mkdir ~/.ssh
    mv ../id_rsa ../id_rsa.pub ~/.ssh
    chmod -R 600 ~/.ssh/*
    echo -e \
      "Host *\n  StrictHostKeyChecking no\n  UserKnownHostsFile=/dev/null" \
      > ~/.ssh/config
    git push -f heroku $(git symbolic-ref --short HEAD):master

You can compare your file with live Rultor configuration of aintshy/hub.

Run It!

badge

Now it’s time to see how it all works. Create a new ticket in the GitHub issue tracker, and post something like this into it (read more about Rultor commands):

@rultor release, tag is `0.1`

You will get a response in a few seconds. The rest will be done by Rultor.

Enjoy :)

BTW, if something doesn’t work as I’ve explained, don’t hesitate to submit a ticket to the Rultor issue tracker. I will try to help you.

PS. I would also recommend to versionalize artifacts through MANIFEST.MF and use jcabi-manifests to read them later.

© Yegor Bugayenko 2014–2018

Simple Java SSH Client

QR code

Simple Java SSH Client

  • comments

An execution of a shell command via SSH can be done in Java, in just a few lines, using jcabi-ssh:

String hello = new Shell.Plain(
  new SSH(
    "ssh.example.com", 22,
    "yegor", "-----BEGIN RSA PRIVATE KEY-----..."
  )
).exec("echo 'Hello, world!'");

jcabi-ssh is a convenient wrapper of JSch, a well-known pure Java implementation of SSH2.

Here is a more complex scenario, where I upload a file via SSH and then read back its grepped content:

Shell shell = new SSH(
  "ssh.example.com", 22,
  "yegor", "-----BEGIN RSA PRIVATE KEY-----..."
);
File file = new File("/tmp/data.txt");
new Shell.Safe(shell).exec(
  "cat > d.txt && grep 'some text' d.txt",
  new FileInputStream(file),
  Logger.stream(Level.INFO, this),
  Logger.stream(Level.WARNING, this)
);

Class SSH, which implements interface Shell, has only one method, exec. This method accepts four arguments:

interface Shell {
  int exec(
    String cmd, InputStream stdin,
    OutputStream stdout, OutputStream stderr
  );
}

I think it’s obvious what these arguments are about.

There are also a few convenient decorators that make it easier to operate with simple commands.

Shell.Safe

Shell.Safe decorates an instance of Shell and throws an exception if the exec exit code is not equal to zero. This may be very useful when you want to make sure that your command executed successfully, but don’t want to duplicate if/throw in many places of your code.

Shell ssh = new Shell.Safe(
  new SSH(
    "ssh.example.com", 22,
    "yegor", "-----BEGIN RSA PRIVATE KEY-----..."
  )
);

Shell.Verbose

Shell.Verbose decorates an instance of Shell and copies stdout and stderr to the slf4j logging facility (using jcabi-log). Of course, you can combine decorators, for example:

Shell ssh = new Shell.Verbose(
  new Shell.Safe(
    new SSH(
      "ssh.example.com", 22,
      "yegor", "-----BEGIN RSA PRIVATE KEY-----..."
    )
  )
);

Shell.Plain

Shell.Plain is a wrapper of Shell that introduces a new exec method with only one argument, a command to execute. It also doesn’t return an exit code, but stdout instead. This should be very convenient when you want to execute a simple command and just get its output (I’m combining it with Shell.Safe for safety):

String login = new Shell.Plain(new Shell.Safe(ssh)).exec("whoami");

Download

You need a single dependency jcabi-ssh.jar in your Maven project (get its latest version in Maven Central):

<dependency>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-ssh</artifactId>
</dependency>

The project is in GitHub. If you have any problems, just submit an issue. I’ll try to help.

An execution of a shell command via SSH can be done in Java, in just a few lines, using jcabi-ssh:

String hello = new Shell.Plain(
  new SSH(
    "ssh.example.com", 22,
    "yegor", "-----BEGIN RSA PRIVATE KEY-----..."
  )
).exec("echo 'Hello, world!'");

jcabi-ssh is a convenient wrapper of JSch, a well-known pure Java implementation of SSH2.

Here is a more complex scenario, where I upload a file via SSH and then read back its grepped content:

Shell shell = new SSH(
  "ssh.example.com", 22,
  "yegor", "-----BEGIN RSA PRIVATE KEY-----..."
);
File file = new File("/tmp/data.txt");
new Shell.Safe(shell).exec(
  "cat > d.txt && grep 'some text' d.txt",
  new FileInputStream(file),
  Logger.stream(Level.INFO, this),
  Logger.stream(Level.WARNING, this)
);

Class SSH, which implements interface Shell, has only one method, exec. This method accepts four arguments:

interface Shell {
  int exec(
    String cmd, InputStream stdin,
    OutputStream stdout, OutputStream stderr
  );
}

I think it’s obvious what these arguments are about.

There are also a few convenient decorators that make it easier to operate with simple commands.

Shell.Safe

Shell.Safe decorates an instance of Shell and throws an exception if the exec exit code is not equal to zero. This may be very useful when you want to make sure that your command executed successfully, but don’t want to duplicate if/throw in many places of your code.

Shell ssh = new Shell.Safe(
  new SSH(
    "ssh.example.com", 22,
    "yegor", "-----BEGIN RSA PRIVATE KEY-----..."
  )
);

Shell.Verbose

Shell.Verbose decorates an instance of Shell and copies stdout and stderr to the slf4j logging facility (using jcabi-log). Of course, you can combine decorators, for example:

Shell ssh = new Shell.Verbose(
  new Shell.Safe(
    new SSH(
      "ssh.example.com", 22,
      "yegor", "-----BEGIN RSA PRIVATE KEY-----..."
    )
  )
);

Shell.Plain

Shell.Plain is a wrapper of Shell that introduces a new exec method with only one argument, a command to execute. It also doesn’t return an exit code, but stdout instead. This should be very convenient when you want to execute a simple command and just get its output (I’m combining it with Shell.Safe for safety):

String login = new Shell.Plain(new Shell.Safe(ssh)).exec("whoami");

Download

You need a single dependency jcabi-ssh.jar in your Maven project (get its latest version in Maven Central):

<dependency>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-ssh</artifactId>
</dependency>

The project is in GitHub. If you have any problems, just submit an issue. I’ll try to help.

© Yegor Bugayenko 2014–2018

How to Deploy to CloudBees, in One Click

QR code

How to Deploy to CloudBees, in One Click

  • comments

When I deploy a new version of stateful.co, a Java web application, to CloudBees, it takes 30 seconds of my time. Maybe even less. Recently, I deployed version 1.6.5. You can see how it all happened, in GitHub issue #6:

The figure

As you see, I gave a command to Rultor, and it packaged, tested and deployed a new version to CloudBees. I didn’t do anything else.

Now let’s see how you can do the same. How you can configure your project so that the deployment of its new version to CloudBees takes just a few seconds of your time.

Since CloudBees is shutting down its PaaS service by the end of December, 2014, this article will have no sense after that.

Configure the CloudBees Maven Plugin

Add this profile to your pom.xml:

<project>
  [..]
  <profiles>
    <profile>
      <id>cloudbees</id>
      <activation>
        <property><name>bees.appId</name></property>
      </activation>
      <pluginRepositories>
        <pluginRepository>
          <id>cloudbees-public-release</id>
          <url>
            http://repository-cloudbees.forge.cloudbees.com/public-release
          </url>
        </pluginRepository>
      </pluginRepositories>
      <build>
        <pluginManagement>
          <plugins>
            <plugin>
              <artifactId>maven-deploy-plugin</artifactId>
              <configuration>
                  <skip>true</skip>
              </configuration>
            </plugin>
          </plugins>
        </pluginManagement>
        <plugins>
          <plugin>
            <groupId>com.cloudbees</groupId>
            <artifactId>bees-maven-plugin</artifactId>
            <version>1.3.2</version>
            <configuration>
              <appid>${bees.id}</appid>
              <apikey>${bees.key}</apikey>
              <secret>${bees.secret}</secret>
            </configuration>
            <executions>
              <execution>
                <id>deploy-to-production</id>
                <phase>deploy</phase>
                <goals>
                  <goal>deploy</goal>
                </goals>
              </execution>
            </executions>
          </plugin>
        </plugins>
      </build>
    </profile>
  </profiles>
</project>

This plugin is not in Maven Central (unfortunately). That’s why we have to specify that <pluginRepository>.

Pay attention to the fact that we’re also disabling maven-deploy-plugin, since it would try to deploy your WAR package to the repository from the <distributionManagement> section. We want to avoid this.

The profile gets activated only when the bees.id property is defined. This won’t happen during your normal development and testing, but it will occur during the deployment cycle initiated by Rultor, because we will define this property in settings.xml (discussed below).

Secure Access to CloudBees

Create an account in CloudBees and register your web application there. CloudBees is free, as long as you don’t need too much computing power. I believe that web applications should be light-weight by definition, so CloudBees’ free layer is an ideal choice.

Create a settings.xml file (but don’t commit it to your repo!):

<settings>
  <profiles>
    <profile>
      <id>cloudbees</id>
      <properties>
        <bees.id>stateful/web</bees.id>
        <bees.key><!-- your key --></bees.key>
        <bees.secret><!-- your secret --></bees.secret>
      </properties>
    </profile>
  </profiles>
</settings>

Encrypt this file using rultor remote:

$ gem install rultor
$ rultor encrypt -p me/test settings.xml

Instead of me/test use the name of your GitHub project.

You should get a settings.xml.asc file; add it to the root directory of your project, commit and push. This file contains your CloudBees credentials, but in an encrypted format. Nobody can read it, except the Rultor server.

Configure Versions Plugin

I recommend using jcabi-parent. It configures the required plugin out-of-the-box. If you’re using it, skip this step.

Otherwise, add this plugin to your pom.xml:

<project>
  [..]
  <build>
    [..]
    <plugins>
      [..]
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>versions-maven-plugin</artifactId>
        <version>2.1</version>
        <configuration>
          <generateBackupPoms>false</generateBackupPoms>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Configure Rultor

Create a .rultor.yml file in the root directory of your project (this reference page explains this format in detail):

decrypt:
  settings.xml: "repo/settings.xml.asc"
release:
  script: |
    mvn versions:set "-DnewVersion=${tag}"
    git commit -am "${tag}"
    mvn clean deploy --settings /home/r/settings.xml

You can compare your file with live Rultor configuration of stateful.co.

Run It!

badge

Now it’s time to see how it all works. Create a new ticket in the GitHub issue tracker, and post something like that into it (read more about Rultor commands):

@rultor release, tag is `0.1`

You will get a response in a few seconds. The rest will be done by Rultor.

Enjoy :)

BTW, if something doesn’t work as I’ve explained, don’t hesitate to submit a ticket to the Rultor issue tracker. I will try to help you.

Also, a similar configuration can be performed for Heroku (using jcabi-heroku-maven-plugin) and for AWS Elastic Beanstalk (using jcabi-beanstalk-maven-plugin). I’ll probably dedicate individual posts to them, as well.

When I deploy a new version of stateful.co, a Java web application, to CloudBees, it takes 30 seconds of my time. Maybe even less. Recently, I deployed version 1.6.5. You can see how it all happened, in GitHub issue #6:

The figure

As you see, I gave a command to Rultor, and it packaged, tested and deployed a new version to CloudBees. I didn’t do anything else.

Now let’s see how you can do the same. How you can configure your project so that the deployment of its new version to CloudBees takes just a few seconds of your time.

Since CloudBees is shutting down its PaaS service by the end of December, 2014, this article will have no sense after that.

Configure the CloudBees Maven Plugin

Add this profile to your pom.xml:

<project>
  [..]
  <profiles>
    <profile>
      <id>cloudbees</id>
      <activation>
        <property><name>bees.appId</name></property>
      </activation>
      <pluginRepositories>
        <pluginRepository>
          <id>cloudbees-public-release</id>
          <url>
            http://repository-cloudbees.forge.cloudbees.com/public-release
          </url>
        </pluginRepository>
      </pluginRepositories>
      <build>
        <pluginManagement>
          <plugins>
            <plugin>
              <artifactId>maven-deploy-plugin</artifactId>
              <configuration>
                  <skip>true</skip>
              </configuration>
            </plugin>
          </plugins>
        </pluginManagement>
        <plugins>
          <plugin>
            <groupId>com.cloudbees</groupId>
            <artifactId>bees-maven-plugin</artifactId>
            <version>1.3.2</version>
            <configuration>
              <appid>${bees.id}</appid>
              <apikey>${bees.key}</apikey>
              <secret>${bees.secret}</secret>
            </configuration>
            <executions>
              <execution>
                <id>deploy-to-production</id>
                <phase>deploy</phase>
                <goals>
                  <goal>deploy</goal>
                </goals>
              </execution>
            </executions>
          </plugin>
        </plugins>
      </build>
    </profile>
  </profiles>
</project>

This plugin is not in Maven Central (unfortunately). That’s why we have to specify that <pluginRepository>.

Pay attention to the fact that we’re also disabling maven-deploy-plugin, since it would try to deploy your WAR package to the repository from the <distributionManagement> section. We want to avoid this.

The profile gets activated only when the bees.id property is defined. This won’t happen during your normal development and testing, but it will occur during the deployment cycle initiated by Rultor, because we will define this property in settings.xml (discussed below).

Secure Access to CloudBees

Create an account in CloudBees and register your web application there. CloudBees is free, as long as you don’t need too much computing power. I believe that web applications should be light-weight by definition, so CloudBees’ free layer is an ideal choice.

Create a settings.xml file (but don’t commit it to your repo!):

<settings>
  <profiles>
    <profile>
      <id>cloudbees</id>
      <properties>
        <bees.id>stateful/web</bees.id>
        <bees.key><!-- your key --></bees.key>
        <bees.secret><!-- your secret --></bees.secret>
      </properties>
    </profile>
  </profiles>
</settings>

Encrypt this file using rultor remote:

$ gem install rultor
$ rultor encrypt -p me/test settings.xml

Instead of me/test use the name of your GitHub project.

You should get a settings.xml.asc file; add it to the root directory of your project, commit and push. This file contains your CloudBees credentials, but in an encrypted format. Nobody can read it, except the Rultor server.

Configure Versions Plugin

I recommend using jcabi-parent. It configures the required plugin out-of-the-box. If you’re using it, skip this step.

Otherwise, add this plugin to your pom.xml:

<project>
  [..]
  <build>
    [..]
    <plugins>
      [..]
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>versions-maven-plugin</artifactId>
        <version>2.1</version>
        <configuration>
          <generateBackupPoms>false</generateBackupPoms>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Configure Rultor

Create a .rultor.yml file in the root directory of your project (this reference page explains this format in detail):

decrypt:
  settings.xml: "repo/settings.xml.asc"
release:
  script: |
    mvn versions:set "-DnewVersion=${tag}"
    git commit -am "${tag}"
    mvn clean deploy --settings /home/r/settings.xml

You can compare your file with live Rultor configuration of stateful.co.

Run It!

badge

Now it’s time to see how it all works. Create a new ticket in the GitHub issue tracker, and post something like that into it (read more about Rultor commands):

@rultor release, tag is `0.1`

You will get a response in a few seconds. The rest will be done by Rultor.

Enjoy :)

BTW, if something doesn’t work as I’ve explained, don’t hesitate to submit a ticket to the Rultor issue tracker. I will try to help you.

Also, a similar configuration can be performed for Heroku (using jcabi-heroku-maven-plugin) and for AWS Elastic Beanstalk (using jcabi-beanstalk-maven-plugin). I’ll probably dedicate individual posts to them, as well.

© Yegor Bugayenko 2014–2018

How to Release to Maven Central, in One Click

QR code

How to Release to Maven Central, in One Click

  • comments

When I release a new version of jcabi-aspects, a Java open source library, to Maven Central, it takes 30 seconds of my time. Maybe even less. Recently, I released version 0.17.2. You can see how it all happened, in GitHub issue #80:

The figure

As you see, I gave a command to Rultor, and it released a new version to Maven central. I didn’t do anything else.

Now let’s see how you can do the same. How you can configure your project so that the release of its new version to Maven Central takes just a few seconds of your time.

By the way, I assume that you’re hosting your project in GitHub. If not, this entire tutorial won’t work. If you are still not in GitHub, I would strongly recommend moving there.

Prepare Your POM

Make sure your pom.xml contains all elements required by Sonatype, explained in Central Sync Requirements. We will deploy to Sonatype, and they will synchronize all JAR (and not only) artifacts to Maven Central.

Register a Project With Sonatype

Create an account in Sonatype JIRA and raise a ticket, asking to approve your groupId. This OSSRH Guide explains this step in more detail.

Create and Distribute a GPG Key

Create a GPG key and distribute it, as explained in this Working with PGP Signatures article.

When this step is done, you should have two files: pubring.gpg and secring.gpg.

Create settings.xml

Create settings.xml, next to the two .gpg files created in the previous step:

<settings>
  <profiles>
    <profile>
      <id>foo</id> <!-- give it the name of your project -->
      <properties>
        <gpg.homedir>/home/r</gpg.homedir>
        <gpg.keyname>9A105525</gpg.keyname>
        <gpg.passphrase>my-secret</gpg.passphrase>
      </properties>
    </profile>
  </profiles>
  <servers>
    <server>
      <id>oss.sonatype.org</id>
      <username><!-- Sonatype JIRA user name --></username>
      <password><!-- Sonatype JIRA pwd --></password>
    </server>
  </servers>
</settings>

In this example, 9A105525 is the ID of your public key, and my-secret is the pass phrase you have used while generating the keys.

Encrypt Security Assets

Now, encrypt these three files with a rultor remote:

$ gem install rultor
$ rultor encrypt -p me/test pubring.gpg
$ rultor encrypt -p me/test secring.gpg
$ rultor encrypt -p me/test settings.xml

Instead of me/test you should use the name of your GitHub project.

You will get three new files: pubring.gpg.asc, secring.gpg.asc and settings.xml.asc. Add them to the root directory of your project, commit and push. The files contain your secret information, but only the Rultor server can decrypt them.

Add Sonatype Repositories

I would recommend using jcabi-parent, as a parent pom for your project. This will make many further steps unnecessary. If you’re using jcabi-parent, skip this step.

However, if you don’t use jcabi-parent, you should add these two repositories to your pom.xml:

<project>
  [...]
  <distributionManagement>
    <repository>
      <id>oss.sonatype.org</id>
      <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
    </repository>
    <snapshotRepository>
      <id>oss.sonatype.org</id>
      <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    </snapshotRepository>
  </distributionManagement>
</project>

Configure GPG Plugin

Again, I’d recommend using jcabi-parent, which configures this plugin automatically. If you’re using it, skip this step.

Otherwise, add this plugin to your pom.xml:

<project>
  [..]
  <build>
    [..]
    <plugins>
      [..]
      <plugin>
        <artifactId>maven-gpg-plugin</artifactId>
        <version>1.5</version>
        <executions>
          <execution>
            <id>sign-artifacts</id>
            <phase>verify</phase>
            <goals>
              <goal>sign</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Configure Versions Plugin

Once again, I recommend using http://parent.jcabi.com. It configures all required plugins out-of-the-box. If you’re using it, skip this step.

Otherwise, add this plugin to your pom.xml:

<project>
  [..]
  <build>
    [..]
    <plugins>
      [..]
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>versions-maven-plugin</artifactId>
        <version>2.1</version>
        <configuration>
          <generateBackupPoms>false</generateBackupPoms>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Configure Sonatype Plugin

Yes, you’re right, http://parent.jcabi.com will help you here as well. If you’re using it, skip this step too.

Otherwise, add these four plugins to your pom.xml:

<project>
  [..]
  <build>
    [..]
    <plugins>
      [..]
      <plugin>
        <artifactId>maven-deploy-plugin</artifactId>
        <configuration>
          <skip>true</skip>
        </configuration>
      </plugin>
      <plugin>
        <artifactId>maven-source-plugin</artifactId>
        <executions>
          <execution>
            <id>package-sources</id>
            <goals>
              <goal>jar</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <artifactId>maven-javadoc-plugin</artifactId>
        <executions>
          <execution>
            <id>package-javadoc</id>
            <phase>package</phase>
            <goals>
              <goal>jar</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.sonatype.plugins</groupId>
        <artifactId>nexus-staging-maven-plugin</artifactId>
        <version>1.6</version>
        <extensions>true</extensions>
        <configuration>
          <serverId>oss.sonatype.org</serverId>
          <nexusUrl>https://oss.sonatype.org/</nexusUrl>
          <description>${project.version}</description>
        </configuration>
        <executions>
          <execution>
            <id>deploy-to-sonatype</id>
            <phase>deploy</phase>
            <goals>
              <goal>deploy</goal>
              <goal>release</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Create Rultor Configuration

Create a .rultor.yml file in the root directory of your project (reference page explains this format in details):

decrypt:
  settings.xml: "repo/settings.xml.asc"
  pubring.gpg: "repo/pubring.gpg.asc"
  secring.gpg: "repo/secring.gpg.asc"
release:
  script: |
    mvn versions:set "-DnewVersion=${tag}"
    git commit -am "${tag}"
    mvn clean deploy -Pjcabi --settings /home/r/settings.xml

You can compare your file with live Rultor configuration of jcabi-aspects.

Run It!

badge

Now it’s time to see how it all works. Create a new ticket in the GitHub issue tracker, and post something like that into it (read more about Rultor commands):

@rultor release, tag is `0.1`

You will get a response in a few seconds. The rest will be done by Rultor.

Enjoy :)

BTW, if something doesn’t work as I’ve explained, don’t hesitate to submit a ticket to Rultor issue tracker. I will try to help you.

Yeah, forgot to mention, Rultor is also doing two important things. First, it creates a GitHub release with a proper description. Second, it posts a tweet about the release, which you can retweet, to make an announcement to your followers. Both features are very convenient for me. For example:

When I release a new version of jcabi-aspects, a Java open source library, to Maven Central, it takes 30 seconds of my time. Maybe even less. Recently, I released version 0.17.2. You can see how it all happened, in GitHub issue #80:

The figure

As you see, I gave a command to Rultor, and it released a new version to Maven central. I didn’t do anything else.

Now let’s see how you can do the same. How you can configure your project so that the release of its new version to Maven Central takes just a few seconds of your time.

By the way, I assume that you’re hosting your project in GitHub. If not, this entire tutorial won’t work. If you are still not in GitHub, I would strongly recommend moving there.

Prepare Your POM

Make sure your pom.xml contains all elements required by Sonatype, explained in Central Sync Requirements. We will deploy to Sonatype, and they will synchronize all JAR (and not only) artifacts to Maven Central.

Register a Project With Sonatype

Create an account in Sonatype JIRA and raise a ticket, asking to approve your groupId. This OSSRH Guide explains this step in more detail.

Create and Distribute a GPG Key

Create a GPG key and distribute it, as explained in this Working with PGP Signatures article.

When this step is done, you should have two files: pubring.gpg and secring.gpg.

Create settings.xml

Create settings.xml, next to the two .gpg files created in the previous step:

<settings>
  <profiles>
    <profile>
      <id>foo</id> <!-- give it the name of your project -->
      <properties>
        <gpg.homedir>/home/r</gpg.homedir>
        <gpg.keyname>9A105525</gpg.keyname>
        <gpg.passphrase>my-secret</gpg.passphrase>
      </properties>
    </profile>
  </profiles>
  <servers>
    <server>
      <id>oss.sonatype.org</id>
      <username><!-- Sonatype JIRA user name --></username>
      <password><!-- Sonatype JIRA pwd --></password>
    </server>
  </servers>
</settings>

In this example, 9A105525 is the ID of your public key, and my-secret is the pass phrase you have used while generating the keys.

Encrypt Security Assets

Now, encrypt these three files with a rultor remote:

$ gem install rultor
$ rultor encrypt -p me/test pubring.gpg
$ rultor encrypt -p me/test secring.gpg
$ rultor encrypt -p me/test settings.xml

Instead of me/test you should use the name of your GitHub project.

You will get three new files: pubring.gpg.asc, secring.gpg.asc and settings.xml.asc. Add them to the root directory of your project, commit and push. The files contain your secret information, but only the Rultor server can decrypt them.

Add Sonatype Repositories

I would recommend using jcabi-parent, as a parent pom for your project. This will make many further steps unnecessary. If you’re using jcabi-parent, skip this step.

However, if you don’t use jcabi-parent, you should add these two repositories to your pom.xml:

<project>
  [...]
  <distributionManagement>
    <repository>
      <id>oss.sonatype.org</id>
      <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
    </repository>
    <snapshotRepository>
      <id>oss.sonatype.org</id>
      <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    </snapshotRepository>
  </distributionManagement>
</project>

Configure GPG Plugin

Again, I’d recommend using jcabi-parent, which configures this plugin automatically. If you’re using it, skip this step.

Otherwise, add this plugin to your pom.xml:

<project>
  [..]
  <build>
    [..]
    <plugins>
      [..]
      <plugin>
        <artifactId>maven-gpg-plugin</artifactId>
        <version>1.5</version>
        <executions>
          <execution>
            <id>sign-artifacts</id>
            <phase>verify</phase>
            <goals>
              <goal>sign</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Configure Versions Plugin

Once again, I recommend using http://parent.jcabi.com. It configures all required plugins out-of-the-box. If you’re using it, skip this step.

Otherwise, add this plugin to your pom.xml:

<project>
  [..]
  <build>
    [..]
    <plugins>
      [..]
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>versions-maven-plugin</artifactId>
        <version>2.1</version>
        <configuration>
          <generateBackupPoms>false</generateBackupPoms>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Configure Sonatype Plugin

Yes, you’re right, http://parent.jcabi.com will help you here as well. If you’re using it, skip this step too.

Otherwise, add these four plugins to your pom.xml:

<project>
  [..]
  <build>
    [..]
    <plugins>
      [..]
      <plugin>
        <artifactId>maven-deploy-plugin</artifactId>
        <configuration>
          <skip>true</skip>
        </configuration>
      </plugin>
      <plugin>
        <artifactId>maven-source-plugin</artifactId>
        <executions>
          <execution>
            <id>package-sources</id>
            <goals>
              <goal>jar</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <artifactId>maven-javadoc-plugin</artifactId>
        <executions>
          <execution>
            <id>package-javadoc</id>
            <phase>package</phase>
            <goals>
              <goal>jar</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.sonatype.plugins</groupId>
        <artifactId>nexus-staging-maven-plugin</artifactId>
        <version>1.6</version>
        <extensions>true</extensions>
        <configuration>
          <serverId>oss.sonatype.org</serverId>
          <nexusUrl>https://oss.sonatype.org/</nexusUrl>
          <description>${project.version}</description>
        </configuration>
        <executions>
          <execution>
            <id>deploy-to-sonatype</id>
            <phase>deploy</phase>
            <goals>
              <goal>deploy</goal>
              <goal>release</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Create Rultor Configuration

Create a .rultor.yml file in the root directory of your project (reference page explains this format in details):

decrypt:
  settings.xml: "repo/settings.xml.asc"
  pubring.gpg: "repo/pubring.gpg.asc"
  secring.gpg: "repo/secring.gpg.asc"
release:
  script: |
    mvn versions:set "-DnewVersion=${tag}"
    git commit -am "${tag}"
    mvn clean deploy -Pjcabi --settings /home/r/settings.xml

You can compare your file with live Rultor configuration of jcabi-aspects.

Run It!

badge

Now it’s time to see how it all works. Create a new ticket in the GitHub issue tracker, and post something like that into it (read more about Rultor commands):

@rultor release, tag is `0.1`

You will get a response in a few seconds. The rest will be done by Rultor.

Enjoy :)

BTW, if something doesn’t work as I’ve explained, don’t hesitate to submit a ticket to Rultor issue tracker. I will try to help you.

Yeah, forgot to mention, Rultor is also doing two important things. First, it creates a GitHub release with a proper description. Second, it posts a tweet about the release, which you can retweet, to make an announcement to your followers. Both features are very convenient for me. For example:

© Yegor Bugayenko 2014–2018

Fluent JDBC Decorator

QR code

Fluent JDBC Decorator

  • comments

badge

This is how you fetch text from a SQL table with jcabi-jdbc:

String name = new JdbcSession(source)
  .sql("SELECT name FROM employee WHERE id = ?")
  .set(1234)
  .select(new SingleOutcome<String>(String.class));

Simple and straight forward, isn’t it? The library simplifies interaction with relational databases via JDBC, avoiding the need to use ORM.

jcabi-jdbc is a lightweight wrapper of JDBC. It is very convenient to use when you don’t need a full-scale ORM (like Hibernate), but want just to select, insert, or update a few rows in a relational database.

Every instance of JdbcSession is a “transaction” in a database. You start it by instantiating the class with a single parameter—data source.

You can obtain the data source from your connection pool. There are many implementations of connection pools. I would recommend that you use BoneCP. Below is an example of how you would connect to PostgreSQL:

@Cacheable(forever = true)
private static DataSource source() {
  BoneCPDataSource src = new BoneCPDataSource();
  src.setDriverClass("org.postgresql.Driver");
  src.setJdbcUrl("jdbc:postgresql://localhost/db_name");
  src.setUser("jeff");
  src.setPassword("secret");
  return src;
}

Be sure to pay attention to the @Cacheable annotation. This post explains how it can help you to cache Java method results for some time. Setting the forever attribute to true means that we don’t want this method to be called more than once. Instead, we want the connection pool to be created just once, and every second call should return its existing instance (kind of like a Singleton pattern).

jcabi-jdbc website explains how you can insert, update, or delete a row. You can also execute any SQL statement.

By default, JdbcSession closes the JDBC connection right after the first select/update/insert operation. Simply put, it is designed to be used mainly for single atomic transactions. However, it is possible to leave the connection open and continue, for example:

new JdbcSession(source)
  .autocommit(false)
  .sql("START TRANSACTION")
  .update()
  .sql("DELETE FROM employee WHERE name = ?")
  .set("Jeff Lebowski")
  .update()
  .sql("INSERT INTO employee VALUES (?)")
  .set("Walter Sobchak")
  .insert(Outcome.VOID)
  .commit();

In this example we’re executing three SQL statements one by one, leaving connection (and transaction) open until commit() is called.

badge

This is how you fetch text from a SQL table with jcabi-jdbc:

String name = new JdbcSession(source)
  .sql("SELECT name FROM employee WHERE id = ?")
  .set(1234)
  .select(new SingleOutcome<String>(String.class));

Simple and straight forward, isn’t it? The library simplifies interaction with relational databases via JDBC, avoiding the need to use ORM.

jcabi-jdbc is a lightweight wrapper of JDBC. It is very convenient to use when you don’t need a full-scale ORM (like Hibernate), but want just to select, insert, or update a few rows in a relational database.

Every instance of JdbcSession is a “transaction” in a database. You start it by instantiating the class with a single parameter—data source.

You can obtain the data source from your connection pool. There are many implementations of connection pools. I would recommend that you use BoneCP. Below is an example of how you would connect to PostgreSQL:

@Cacheable(forever = true)
private static DataSource source() {
  BoneCPDataSource src = new BoneCPDataSource();
  src.setDriverClass("org.postgresql.Driver");
  src.setJdbcUrl("jdbc:postgresql://localhost/db_name");
  src.setUser("jeff");
  src.setPassword("secret");
  return src;
}

Be sure to pay attention to the @Cacheable annotation. This post explains how it can help you to cache Java method results for some time. Setting the forever attribute to true means that we don’t want this method to be called more than once. Instead, we want the connection pool to be created just once, and every second call should return its existing instance (kind of like a Singleton pattern).

jcabi-jdbc website explains how you can insert, update, or delete a row. You can also execute any SQL statement.

By default, JdbcSession closes the JDBC connection right after the first select/update/insert operation. Simply put, it is designed to be used mainly for single atomic transactions. However, it is possible to leave the connection open and continue, for example:

new JdbcSession(source)
  .autocommit(false)
  .sql("START TRANSACTION")
  .update()
  .sql("DELETE FROM employee WHERE name = ?")
  .set("Jeff Lebowski")
  .update()
  .sql("INSERT INTO employee VALUES (?)")
  .set("Walter Sobchak")
  .insert(Outcome.VOID)
  .commit();

In this example we’re executing three SQL statements one by one, leaving connection (and transaction) open until commit() is called.

© Yegor Bugayenko 2014–2018

How to Retry Java Method Call on Exception

QR code

How to Retry Java Method Call on Exception

  • comments

badge

If you have a method that fails occasionally and you want to retry it a few times before throwing an exception. @RetryOnFailure from jcabi-aspects can help. For example, if you’re downloading the following web page:

@RetryOnFailure(
  attempts = 3,
  delay = 10,
  unit = TimeUnit.SECONDS
)
public String load(URL url) {
  return url.openConnection().getContent();
}

This method call will throw an exception only after three failed executions with a ten seconds interval between them.

This post explains how jcabi-aspects works with binary weaving. This mechanism integrates AspectJ with your code.

When method load() from the example above is called, this is what is happening behind the scene (pseudo-code):

while (attempts++ < 3) {
  try {
    return original_load(url);
  } catch (Throwable ex) {
    log("we failed, will try again in 10 seconds");
    sleep(10);
  }
}

This approach may be very useful in the following situations (based on my experience):

  • Executing JDBC SELECT statements

  • Loading data from HTTP, S3, FTP, etc resources

  • Uploading data over the network

  • Fetching data through RESTful stateless API-s

The project is in GitHub.

badge

If you have a method that fails occasionally and you want to retry it a few times before throwing an exception. @RetryOnFailure from jcabi-aspects can help. For example, if you’re downloading the following web page:

@RetryOnFailure(
  attempts = 3,
  delay = 10,
  unit = TimeUnit.SECONDS
)
public String load(URL url) {
  return url.openConnection().getContent();
}

This method call will throw an exception only after three failed executions with a ten seconds interval between them.

This post explains how jcabi-aspects works with binary weaving. This mechanism integrates AspectJ with your code.

When method load() from the example above is called, this is what is happening behind the scene (pseudo-code):

while (attempts++ < 3) {
  try {
    return original_load(url);
  } catch (Throwable ex) {
    log("we failed, will try again in 10 seconds");
    sleep(10);
  }
}

This approach may be very useful in the following situations (based on my experience):

  • Executing JDBC SELECT statements

  • Loading data from HTTP, S3, FTP, etc resources

  • Uploading data over the network

  • Fetching data through RESTful stateless API-s

The project is in GitHub.

© Yegor Bugayenko 2014–2018

Strict Control of Java Code Quality

QR code

Strict Control of Java Code Quality

  • comments

There are many tools that control the quality of Java code, including Checkstyle, PMD, FindBugs, Cobertura, etc. All of them are usually used to analyze quality and build some fancy reports. Very often, those reports are published by continuous integration servers, like Jenkins.

Ratatouille (2007) by Brad Bird and Jan Pinkava
Ratatouille (2007) by Brad Bird and Jan Pinkava

Qulice takes things one step further. It aggregates a few quality checkers, configures them to a maximum strict mode, and breaks your build if any of them fail.

badge

Seriously. There are over 130 checks in Checkstyle, over 400 rules in PMD, and over 400 bugs in FindBugs. All of them should say: “Yes, we like your code.” Otherwise, your build shouldn’t pass.

What do you think? Would it be convenient for you—to have your code rejected every time it breaks just one of 900 checks? Would it be productive for the team—to force developers to focus so much on code quality?

First Reaction

If you join one of our teams as a Java developer, you will develop your code in branches and, then, Rultor will merge your changes into master. Before actually merging, though, Rultor will run an automated build script to make sure that your branch doesn’t break it.

As a static analysis tool, Qulice is just one of the steps in the automated build script. It is actually a Maven plugin and we automate Java builds with Maven 3x. Thus, if your changes break any of Qulice’s rules, your entire branch gets rejected.

Your first reaction—I’ve seen it hundreds of times—will be negative. You may actually become frustrated enough to leave the project immediately. You may say something like this (I’m quoting real life stories):

  • “These quality rules entirely ruin my creativity!”

  • “Instead of wasting time on these misplaced commas and braces, we’d be better off developing new features!”

  • “I’ve done many successful projects in my life, never heard about this ridiculous quality checking…”

This first reaction is only logical. I’ve seen many people say things like this, in both open source and commercial projects. Not only in Java, but also in PHP (with phpcs and phpmd) and Ruby (with rubocop and simplecov).

How do I answer? Read on.

On Second Thought

My experience tells me that the sooner someone can get used to the strict quality control of Qulice, the faster he/she can learn and grow; the better programmer he/she is; and the further he/she can go with us and our projects.

Having this experience in mind, I recommend that all new project members be patient and try to get used to this new approach to quality. In a few weeks, those who stick with it start to understand why this approach is good for the project and for them, as Java engineers.

Why is it good? Read on.

What Do Projects Get From Qulice?

Let’s take one simple rule as an example. Here is a piece of Java code that Qulice would complain about (due to the DesignForExtension rule from Checkstyle):

public class Employee {
  public String name() {
    return "Jeff";
  }
}

What is wrong with this code? Method name() is not final and can be overridden by a class that extends Employee. Design-wise this is wrong, since a child class is allowed to break a super class, overriding its method.

What is the right design? This one:

public class Employee {
  public final String name() {
    return "Jeff";
  }
}

Now, the method is final and can’t be overridden by child classes. It is a much safer design (according to Checkstyle, and I agree).

So, let’s say we make this rule mandatory for all classes in the project. What does the project gain from this? It can promise its members (programmers) a higher quality of work, compared to other projects that don’t have this restriction, mostly because of:

  • Predictability of Design—I don’t have to scroll through the entire class to make sure it doesn’t have methods that can be accidentally overridden. I know for sure that this can’t happen in this project. In other words, I know what to expect.

  • Less Hidden Tricks—Higher predictability of design leads to better visibility of mistakes and tricks. Standardization of source code makes it uniform. This means that it’s easier to read and spot problems.

  • Industry Standards—The decision to use this design is made by Checkstyle, not by a project architect. For me, as a project developer, this means that I’m following industry standards. That makes the project (and its leaders) more respectable.

  • Learning—I’ll bet that most of you who read this post didn’t know about the design rule explained above. Just by reading this article, you learned something new. Imagine how much you could learn after making your code compliant to all 900 rules of Qulice (Checkstyle + PMD + FindBugs).

The point about learning brings us to the last, and the most important, thought to discuss.

What Do I Get from Qulice?

As a programmer, I hope you already realize what you get from working in a project that raises its quality bar as high as Qulice asks. Yes, you’ll learn a lot of new things about writing quality Java code.

On top of that though, I would actually say that you are getting free lessons with every new line of code you write. And the teacher is a software, written by hundreds of Java developers, for the last ten years. Qulice just integrates those software tools together. Truthfully, it is the developers who are the real authors of quality checks and rules.

So, what do I tell those who complain about quality rules being too strict? I say this: “Do you want to learn and improve, or do you just want to get paid and get away with it?”

ps. You can use my settings.jar for IntelliJ, they are rather strict and will help you clean your code even before Qulice starts to complain.

There are many tools that control the quality of Java code, including Checkstyle, PMD, FindBugs, Cobertura, etc. All of them are usually used to analyze quality and build some fancy reports. Very often, those reports are published by continuous integration servers, like Jenkins.

Ratatouille (2007) by Brad Bird and Jan Pinkava
Ratatouille (2007) by Brad Bird and Jan Pinkava

Qulice takes things one step further. It aggregates a few quality checkers, configures them to a maximum strict mode, and breaks your build if any of them fail.

badge

Seriously. There are over 130 checks in Checkstyle, over 400 rules in PMD, and over 400 bugs in FindBugs. All of them should say: “Yes, we like your code.” Otherwise, your build shouldn’t pass.

What do you think? Would it be convenient for you—to have your code rejected every time it breaks just one of 900 checks? Would it be productive for the team—to force developers to focus so much on code quality?

First Reaction

If you join one of our teams as a Java developer, you will develop your code in branches and, then, Rultor will merge your changes into master. Before actually merging, though, Rultor will run an automated build script to make sure that your branch doesn’t break it.

As a static analysis tool, Qulice is just one of the steps in the automated build script. It is actually a Maven plugin and we automate Java builds with Maven 3x. Thus, if your changes break any of Qulice’s rules, your entire branch gets rejected.

Your first reaction—I’ve seen it hundreds of times—will be negative. You may actually become frustrated enough to leave the project immediately. You may say something like this (I’m quoting real life stories):

  • “These quality rules entirely ruin my creativity!”

  • “Instead of wasting time on these misplaced commas and braces, we’d be better off developing new features!”

  • “I’ve done many successful projects in my life, never heard about this ridiculous quality checking…”

This first reaction is only logical. I’ve seen many people say things like this, in both open source and commercial projects. Not only in Java, but also in PHP (with phpcs and phpmd) and Ruby (with rubocop and simplecov).

How do I answer? Read on.

On Second Thought

My experience tells me that the sooner someone can get used to the strict quality control of Qulice, the faster he/she can learn and grow; the better programmer he/she is; and the further he/she can go with us and our projects.

Having this experience in mind, I recommend that all new project members be patient and try to get used to this new approach to quality. In a few weeks, those who stick with it start to understand why this approach is good for the project and for them, as Java engineers.

Why is it good? Read on.

What Do Projects Get From Qulice?

Let’s take one simple rule as an example. Here is a piece of Java code that Qulice would complain about (due to the DesignForExtension rule from Checkstyle):

public class Employee {
  public String name() {
    return "Jeff";
  }
}

What is wrong with this code? Method name() is not final and can be overridden by a class that extends Employee. Design-wise this is wrong, since a child class is allowed to break a super class, overriding its method.

What is the right design? This one:

public class Employee {
  public final String name() {
    return "Jeff";
  }
}

Now, the method is final and can’t be overridden by child classes. It is a much safer design (according to Checkstyle, and I agree).

So, let’s say we make this rule mandatory for all classes in the project. What does the project gain from this? It can promise its members (programmers) a higher quality of work, compared to other projects that don’t have this restriction, mostly because of:

  • Predictability of Design—I don’t have to scroll through the entire class to make sure it doesn’t have methods that can be accidentally overridden. I know for sure that this can’t happen in this project. In other words, I know what to expect.

  • Less Hidden Tricks—Higher predictability of design leads to better visibility of mistakes and tricks. Standardization of source code makes it uniform. This means that it’s easier to read and spot problems.

  • Industry Standards—The decision to use this design is made by Checkstyle, not by a project architect. For me, as a project developer, this means that I’m following industry standards. That makes the project (and its leaders) more respectable.

  • Learning—I’ll bet that most of you who read this post didn’t know about the design rule explained above. Just by reading this article, you learned something new. Imagine how much you could learn after making your code compliant to all 900 rules of Qulice (Checkstyle + PMD + FindBugs).

The point about learning brings us to the last, and the most important, thought to discuss.

What Do I Get from Qulice?

As a programmer, I hope you already realize what you get from working in a project that raises its quality bar as high as Qulice asks. Yes, you’ll learn a lot of new things about writing quality Java code.

On top of that though, I would actually say that you are getting free lessons with every new line of code you write. And the teacher is a software, written by hundreds of Java developers, for the last ten years. Qulice just integrates those software tools together. Truthfully, it is the developers who are the real authors of quality checks and rules.

So, what do I tell those who complain about quality rules being too strict? I say this: “Do you want to learn and improve, or do you just want to get paid and get away with it?”

ps. You can use my settings.jar for IntelliJ, they are rather strict and will help you clean your code even before Qulice starts to complain.

© Yegor Bugayenko 2014–2018

Cache Java Method Results

QR code

Cache Java Method Results

  • comments

badge

Say, you have a method that takes time to execute and you want its result to be cached. There are many solutions, including Apache Commons JCS, Ehcache, JSR 107, Guava Caching and many others.

jcabi-aspects offers a very simple one, based on AOP aspects and Java6 annotations:

import com.jcabi.aspects.Cacheable;
public class Page {
  @Cacheable(lifetime = 5, unit = TimeUnit.MINUTES)
  String load() {
    return new URL("http://google.com").getContent().toString();
  }
}

The result of load() method will be cached in memory for five minutes.

How It Works?

This post about AOP, AspectJ and method logging explains how “aspect weaving” works (I highly recommend that you read it first).

Here I’ll explain how caching works.

The approach is very straight forward. There is a static hash map with keys as “method coordinates” and values as their results. Method coordinates consist of the object, an owner of the method and a method name with parameter types.

In the example above, right after the method load() finishes, the map gets a new entry (simplified example, of course):

key: [page, "load()"]
value: "<html>...</html>"

Every consecutive call to load() will be intercepted by the aspect from jcabi-aspects and resolved immediately with a value from the cache map. The method will not get any control until the end of its lifetime, which is five minutes in the example above.

What About Cache Flushing?

Sometimes it’s necessary to have the ability to flush cache before the end of its lifetime. Here is a practical example:

import com.jcabi.aspects.Cacheable;
public class Employees {
  @Cacheable(lifetime = 1, unit = TimeUnit.HOURS)
  int size() {
    // calculate their amount in MySQL
  }
  @Cacheable.FlushBefore
  void add(Employee employee) {
    // add a new one to MySQL
  }
}

It’s obvious that the number of employees in the database will be different after add() method execution and the result of size() should be invalidated in cache. This invalidation operation is called “flushing” and @Cacheable.FlushBefore triggers it.

Actually, every call to add() invalidates all cached methods in this class, not only size().

There is also @Cacheable.FlushAfter. The difference is that FlushBefore guarantees that cache is already invalidated when the method add() starts. FlushAfter invalidates cache after method add() finishes. This small difference makes a big one, sometimes.

This article explains how to add jcabi-aspects to your project.

badge

Say, you have a method that takes time to execute and you want its result to be cached. There are many solutions, including Apache Commons JCS, Ehcache, JSR 107, Guava Caching and many others.

jcabi-aspects offers a very simple one, based on AOP aspects and Java6 annotations:

import com.jcabi.aspects.Cacheable;
public class Page {
  @Cacheable(lifetime = 5, unit = TimeUnit.MINUTES)
  String load() {
    return new URL("http://google.com").getContent().toString();
  }
}

The result of load() method will be cached in memory for five minutes.

How It Works?

This post about AOP, AspectJ and method logging explains how “aspect weaving” works (I highly recommend that you read it first).

Here I’ll explain how caching works.

The approach is very straight forward. There is a static hash map with keys as “method coordinates” and values as their results. Method coordinates consist of the object, an owner of the method and a method name with parameter types.

In the example above, right after the method load() finishes, the map gets a new entry (simplified example, of course):

key: [page, "load()"]
value: "<html>...</html>"

Every consecutive call to load() will be intercepted by the aspect from jcabi-aspects and resolved immediately with a value from the cache map. The method will not get any control until the end of its lifetime, which is five minutes in the example above.

What About Cache Flushing?

Sometimes it’s necessary to have the ability to flush cache before the end of its lifetime. Here is a practical example:

import com.jcabi.aspects.Cacheable;
public class Employees {
  @Cacheable(lifetime = 1, unit = TimeUnit.HOURS)
  int size() {
    // calculate their amount in MySQL
  }
  @Cacheable.FlushBefore
  void add(Employee employee) {
    // add a new one to MySQL
  }
}

It’s obvious that the number of employees in the database will be different after add() method execution and the result of size() should be invalidated in cache. This invalidation operation is called “flushing” and @Cacheable.FlushBefore triggers it.

Actually, every call to add() invalidates all cached methods in this class, not only size().

There is also @Cacheable.FlushAfter. The difference is that FlushBefore guarantees that cache is already invalidated when the method add() starts. FlushAfter invalidates cache after method add() finishes. This small difference makes a big one, sometimes.

This article explains how to add jcabi-aspects to your project.

© Yegor Bugayenko 2014–2018

Liquibase with Maven

QR code

Liquibase with Maven

Liquibase is a migration management tool for relational databases. It versionalizes schema and data changes in a database; similar to the way Git or SVN works for source code. Thanks to their Maven plugin, Liquibase can be used as a part of a build automation scenario.

Maven Plugin

Let’s assume you’re using MySQL (PostgreSQL or any other database configuration will be very similar.)

Add liquibase-maven-plugin to your pom.xml (get its latest version in Maven Central):

<project>
  [...]
  <build>
    [...]
    <plugins>
      <plugin>
        <groupId>org.liquibase</groupId>
        <artifactId>liquibase-maven-plugin</artifactId>
        <configuration>
          <changeLogFile>
            ${basedir}/src/main/liquibase/master.xml
          </changeLogFile>
          <driver>com.mysql.jdbc.Driver</driver>
          <url>jdbc:mysql://${mysql.host}:${mysql.port}/${mysql.db}</url>
          <username>${mysql.login}</username>
          <password>${mysql.password}</password>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

To check that it works, run mvn liquibase:help.

I would recommend you keep database credentials in settings.xml and in their respective profiles. For example:

<settings>
  <profiles>
    <profile>
      <id>production</id>
      <properties>
        <mysql.host>db.example.com</mysql.host>
        <mysql.port>3306</mysql.port>
        <mysql.db>example</mysql.db>
      </properties>
    </profile>
    <profile>
      <id>test</id>
      <properties>
        <mysql.host>test-db.example.com</mysql.host>
        <mysql.port>3306</mysql.port>
        <mysql.db>example-db</mysql.db>
      </properties>
    </profile>
  </profiles>
</settings>

When you run Maven, don’t forget to turn on one of the profiles. For example: mvn -Pproduction.

Initial Schema

I assume you already have a database with a schema (tables, triggers, views, etc.) and some data. You should “reverse engineer” it and create an initial schema file for Liquibase. In other words, we should inform Liquibase where we are at the moment, so that it starts to apply changes from this point.

Maven plugin doesn’t support it, so you will have to run Liquibase directly. But, it’s not that difficult. First, run mvn liquibase:help in order to download all artifacts. Then, replace placeholders with your actual credentials:

$ java -jar \
  ~/.m2/repository/org/liquibase/liquibase-core/3.1.1/liquibase-core-3.1.1.jar \
  --driver=com.mysql.jdbc.Driver \
  --url=jdbc:mysql://db.example.com:3306/example \
  --username=example --password=example \
  generateChangeLog > src/main/liquibase/2014/000-initial-schema.xml

Liquibase will analyze your current database schema and copy its own schema into src/main/liquibase/2014/000-initial-schema.xml.

Master Changeset

Now, create XML master changeset and save it to src/main/liquibase/master.xml:

<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
    http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-2.0.xsd">
  <includeAll path="src/main/liquibase/2014" />
</databaseChangeLog>

It is an entry point for Liquibase. It starts from this file and loads all other changesets available in src/main/liquibase/2014. They should be either .xml or .sql. I recommend that you use XML mostly because it is easier to maintain and works faster.

Incremental Changesets

Let’s create a simple changeset, which adds a new column to an existing table:

<databaseChangeLog xmlns='http://www.liquibase.org/xml/ns/dbchangelog'
  xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
  xsi:schemaLocation='http://www.liquibase.org/xml/ns/dbchangelog
    http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-2.0.xsd'>
  <changeSet id="002" author="Yegor">
    <sql>
      ALTER TABLE user ADD COLUMN address VARCHAR(1024);
    </sql>
  </changeSet>
</databaseChangeLog>

We save this file in src/main/liquibase/2014/002-add-user-address.xml. In big projects, you can name your files by the names of the tickets they are produced in. For example, 045-3432.xml, which means changeset number 45 coming from ticket #3432.

The important thing is to have this numeric prefix in front of file names, in order to sort them correctly. We want changes to be applied in their correct chronological order.

That’s it. We’re ready to run mvn liquibase:update -Pproduction and our production database will be updated—a new column will be added to the user table.

Also, see how MySQL Maven Plugin can help you to automate integration testing of database-connected classes.

Liquibase is a migration management tool for relational databases. It versionalizes schema and data changes in a database; similar to the way Git or SVN works for source code. Thanks to their Maven plugin, Liquibase can be used as a part of a build automation scenario.

Maven Plugin

Let’s assume you’re using MySQL (PostgreSQL or any other database configuration will be very similar.)

Add liquibase-maven-plugin to your pom.xml (get its latest version in Maven Central):

<project>
  [...]
  <build>
    [...]
    <plugins>
      <plugin>
        <groupId>org.liquibase</groupId>
        <artifactId>liquibase-maven-plugin</artifactId>
        <configuration>
          <changeLogFile>
            ${basedir}/src/main/liquibase/master.xml
          </changeLogFile>
          <driver>com.mysql.jdbc.Driver</driver>
          <url>jdbc:mysql://${mysql.host}:${mysql.port}/${mysql.db}</url>
          <username>${mysql.login}</username>
          <password>${mysql.password}</password>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

To check that it works, run mvn liquibase:help.

I would recommend you keep database credentials in settings.xml and in their respective profiles. For example:

<settings>
  <profiles>
    <profile>
      <id>production</id>
      <properties>
        <mysql.host>db.example.com</mysql.host>
        <mysql.port>3306</mysql.port>
        <mysql.db>example</mysql.db>
      </properties>
    </profile>
    <profile>
      <id>test</id>
      <properties>
        <mysql.host>test-db.example.com</mysql.host>
        <mysql.port>3306</mysql.port>
        <mysql.db>example-db</mysql.db>
      </properties>
    </profile>
  </profiles>
</settings>

When you run Maven, don’t forget to turn on one of the profiles. For example: mvn -Pproduction.

Initial Schema

I assume you already have a database with a schema (tables, triggers, views, etc.) and some data. You should “reverse engineer” it and create an initial schema file for Liquibase. In other words, we should inform Liquibase where we are at the moment, so that it starts to apply changes from this point.

Maven plugin doesn’t support it, so you will have to run Liquibase directly. But, it’s not that difficult. First, run mvn liquibase:help in order to download all artifacts. Then, replace placeholders with your actual credentials:

$ java -jar \
  ~/.m2/repository/org/liquibase/liquibase-core/3.1.1/liquibase-core-3.1.1.jar \
  --driver=com.mysql.jdbc.Driver \
  --url=jdbc:mysql://db.example.com:3306/example \
  --username=example --password=example \
  generateChangeLog > src/main/liquibase/2014/000-initial-schema.xml

Liquibase will analyze your current database schema and copy its own schema into src/main/liquibase/2014/000-initial-schema.xml.

Master Changeset

Now, create XML master changeset and save it to src/main/liquibase/master.xml:

<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
    http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-2.0.xsd">
  <includeAll path="src/main/liquibase/2014" />
</databaseChangeLog>

It is an entry point for Liquibase. It starts from this file and loads all other changesets available in src/main/liquibase/2014. They should be either .xml or .sql. I recommend that you use XML mostly because it is easier to maintain and works faster.

Incremental Changesets

Let’s create a simple changeset, which adds a new column to an existing table:

<databaseChangeLog xmlns='http://www.liquibase.org/xml/ns/dbchangelog'
  xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
  xsi:schemaLocation='http://www.liquibase.org/xml/ns/dbchangelog
    http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-2.0.xsd'>
  <changeSet id="002" author="Yegor">
    <sql>
      ALTER TABLE user ADD COLUMN address VARCHAR(1024);
    </sql>
  </changeSet>
</databaseChangeLog>

We save this file in src/main/liquibase/2014/002-add-user-address.xml. In big projects, you can name your files by the names of the tickets they are produced in. For example, 045-3432.xml, which means changeset number 45 coming from ticket #3432.

The important thing is to have this numeric prefix in front of file names, in order to sort them correctly. We want changes to be applied in their correct chronological order.

That’s it. We’re ready to run mvn liquibase:update -Pproduction and our production database will be updated—a new column will be added to the user table.

Also, see how MySQL Maven Plugin can help you to automate integration testing of database-connected classes.

© Yegor Bugayenko 2014–2018

How to Read MANIFEST.MF Files

QR code

How to Read MANIFEST.MF Files

badge

Every Java package (JAR, WAR, EAR, etc.) has a MANIFEST.MF file in the META-INF directory. The file contains a list of attributes, which describe this particular package. For example:

Manifest-Version: 1.0
Created-By: 1.7.0_06 (Oracle Corporation)
Main-Class: MyPackage.MyClass

When your application has multiple JAR dependencies, you have multiple MANIFEST.MF files in your class path. All of them have the same location: META-INF/MANIFEST.MF. Very often it is necessary to go through all of them in runtime and find the attribute by its name.

jcabi-manifests makes it possible with a one-liner:

import com.jcabi.manifests.Manifests;
String created = Manifests.read("Created-By");

Let’s see why you would want to read attributes from manifest files, and how it works on a low level.

Package Versioning

When you package a library or even a web application, it is a good practice to add an attribute to its MANIFEST.MF with the package version name and build number. In Maven, maven-jar-plugin can help you (almost the same configuration for maven-war-plugin):

<plugin>
  <artifactId>maven-jar-plugin</artifactId>
  <configuration>
    <archive>
      <manifestEntries>
        <Foo-Version>${project.version}</Foo-Version>
        <Foo-Hash>${buildNumber}</Foo-Hash>
      </manifestEntries>
    </archive>
  </configuration>
</plugin>

buildnumber-maven-plugin will help you to get ${buildNumber} from Git, SVN or Mercurial:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>buildnumber-maven-plugin</artifactId>
  <executions>
    <execution>
      <goals>
        <goal>create</goal>
      </goals>
    </execution>
  </executions>
</plugin>

After all these manipulations, MANIFEST.MF, in your JAR will contain these two extra lines (on top of all others added there by Maven by default):

Foo-Version: 1.0-SNAPSHOT
Foo-Hash: 7ef4ac3

In runtime, you can show these values to the user to help him understand which version of the product he is working with at any given moment.

Look at stateful.co, for example. At the bottom of its front page, you see the version number and Git hash. They are retrieved from MANIFEST.MF of the deployed WAR package, on every page click.

Credentials

Although this may be considered as a bad practice (see Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation by Jez Humble and David Farley), sometimes it is convenient to package production credentials right into the JAR/WAR archive during the continuous integration/delivery cycle.

For example, you can encode your PostgreSQL connection details right into MANIFEST.MF:

<plugin>
  <artifactId>maven-war-plugin</artifactId>
  <configuration>
    <archive>
      <manifestEntries>
        <Pgsql>jdbc:postgresql://${pg.host}:${pg.port}/${pg.db}</Pgsql>
      </manifestEntries>
    </archive>
  </configuration>
</plugin>

Afterwards, you can retrieve them in runtime using jcabi-manifests:

String url = Manifests.read("Pgsql");

If you know of any other useful purposes for MANIFEST.MF, let me know :)

badge

Every Java package (JAR, WAR, EAR, etc.) has a MANIFEST.MF file in the META-INF directory. The file contains a list of attributes, which describe this particular package. For example:

Manifest-Version: 1.0
Created-By: 1.7.0_06 (Oracle Corporation)
Main-Class: MyPackage.MyClass

When your application has multiple JAR dependencies, you have multiple MANIFEST.MF files in your class path. All of them have the same location: META-INF/MANIFEST.MF. Very often it is necessary to go through all of them in runtime and find the attribute by its name.

jcabi-manifests makes it possible with a one-liner:

import com.jcabi.manifests.Manifests;
String created = Manifests.read("Created-By");

Let’s see why you would want to read attributes from manifest files, and how it works on a low level.

Package Versioning

When you package a library or even a web application, it is a good practice to add an attribute to its MANIFEST.MF with the package version name and build number. In Maven, maven-jar-plugin can help you (almost the same configuration for maven-war-plugin):

<plugin>
  <artifactId>maven-jar-plugin</artifactId>
  <configuration>
    <archive>
      <manifestEntries>
        <Foo-Version>${project.version}</Foo-Version>
        <Foo-Hash>${buildNumber}</Foo-Hash>
      </manifestEntries>
    </archive>
  </configuration>
</plugin>

buildnumber-maven-plugin will help you to get ${buildNumber} from Git, SVN or Mercurial:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>buildnumber-maven-plugin</artifactId>
  <executions>
    <execution>
      <goals>
        <goal>create</goal>
      </goals>
    </execution>
  </executions>
</plugin>

After all these manipulations, MANIFEST.MF, in your JAR will contain these two extra lines (on top of all others added there by Maven by default):

Foo-Version: 1.0-SNAPSHOT
Foo-Hash: 7ef4ac3

In runtime, you can show these values to the user to help him understand which version of the product he is working with at any given moment.

Look at stateful.co, for example. At the bottom of its front page, you see the version number and Git hash. They are retrieved from MANIFEST.MF of the deployed WAR package, on every page click.

Credentials

Although this may be considered as a bad practice (see Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation by Jez Humble and David Farley), sometimes it is convenient to package production credentials right into the JAR/WAR archive during the continuous integration/delivery cycle.

For example, you can encode your PostgreSQL connection details right into MANIFEST.MF:

<plugin>
  <artifactId>maven-war-plugin</artifactId>
  <configuration>
    <archive>
      <manifestEntries>
        <Pgsql>jdbc:postgresql://${pg.host}:${pg.port}/${pg.db}</Pgsql>
      </manifestEntries>
    </archive>
  </configuration>
</plugin>

Afterwards, you can retrieve them in runtime using jcabi-manifests:

String url = Manifests.read("Pgsql");

If you know of any other useful purposes for MANIFEST.MF, let me know :)

© Yegor Bugayenko 2014–2018

SASS in Java Webapp

QR code

SASS in Java Webapp

SASS is a powerful and very popular language for writing CSS style sheets. This is how I’m using SASS in my Maven projects.

First, I change the extensions of .css files to .scss and move them from src/main/webapp/css to src/main/scss.

Then, I configure the sass-maven-plugin (get its latest versions in Maven Central):

<plugin>
  <groupId>nl.geodienstencentrum.maven</groupId>
  <artifactId>sass-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>generate-css</id>
      <phase>generate-resources</phase>
      <goals>
        <goal>update-stylesheets</goal>
      </goals>
      <configuration>
        <sassSourceDirectory>${basedir}/src/main/scss</sassSourceDirectory>
        <destination>${project.build.directory}/css</destination>
      </configuration>
    </execution>
  </executions>
</plugin>

The SASS compiler will compile .scss files from src/main/scss and place .css files into target/css.

Then, I configure the minify-maven-plugin to compress/minify the style sheets produced by the SASS compiler:

<plugin>
  <groupId>com.samaxes.maven</groupId>
  <artifactId>minify-maven-plugin</artifactId>
  <configuration>
    <charset>UTF-8</charset>
    <nosuffix>true</nosuffix>
    <webappTargetDir>${project.build.directory}/css-min</webappTargetDir>
  </configuration>
  <executions>
    <execution>
      <id>minify-css</id>
      <goals>
        <goal>minify</goal>
      </goals>
      <configuration>
        <webappSourceDir>${project.build.directory}</webappSourceDir>
        <cssSourceDir>css</cssSourceDir>
        <cssSourceIncludes>
          <include>*.css</include>
        </cssSourceIncludes>
        <skipMerge>true</skipMerge>
      </configuration>
    </execution>
  </executions>
</plugin>

Minified .css files will be placed into target/css-min.

The final step is to configure the maven-war-plugin to pick up .css files and package them into the final WAR archive:

<plugin>
  <artifactId>maven-war-plugin</artifactId>
  <configuration>
    [..other configuration options..]
    <webResources combine.children="append">
      <resource>
        <directory>${project.build.directory}/css-min</directory>
      </resource>
    </webResources>
  </configuration>
</plugin>

That’s it.

SASS is a powerful and very popular language for writing CSS style sheets. This is how I’m using SASS in my Maven projects.

First, I change the extensions of .css files to .scss and move them from src/main/webapp/css to src/main/scss.

Then, I configure the sass-maven-plugin (get its latest versions in Maven Central):

<plugin>
  <groupId>nl.geodienstencentrum.maven</groupId>
  <artifactId>sass-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>generate-css</id>
      <phase>generate-resources</phase>
      <goals>
        <goal>update-stylesheets</goal>
      </goals>
      <configuration>
        <sassSourceDirectory>${basedir}/src/main/scss</sassSourceDirectory>
        <destination>${project.build.directory}/css</destination>
      </configuration>
    </execution>
  </executions>
</plugin>

The SASS compiler will compile .scss files from src/main/scss and place .css files into target/css.

Then, I configure the minify-maven-plugin to compress/minify the style sheets produced by the SASS compiler:

<plugin>
  <groupId>com.samaxes.maven</groupId>
  <artifactId>minify-maven-plugin</artifactId>
  <configuration>
    <charset>UTF-8</charset>
    <nosuffix>true</nosuffix>
    <webappTargetDir>${project.build.directory}/css-min</webappTargetDir>
  </configuration>
  <executions>
    <execution>
      <id>minify-css</id>
      <goals>
        <goal>minify</goal>
      </goals>
      <configuration>
        <webappSourceDir>${project.build.directory}</webappSourceDir>
        <cssSourceDir>css</cssSourceDir>
        <cssSourceIncludes>
          <include>*.css</include>
        </cssSourceIncludes>
        <skipMerge>true</skipMerge>
      </configuration>
    </execution>
  </executions>
</plugin>

Minified .css files will be placed into target/css-min.

The final step is to configure the maven-war-plugin to pick up .css files and package them into the final WAR archive:

<plugin>
  <artifactId>maven-war-plugin</artifactId>
  <configuration>
    [..other configuration options..]
    <webResources combine.children="append">
      <resource>
        <directory>${project.build.directory}/css-min</directory>
      </resource>
    </webResources>
  </configuration>
</plugin>

That’s it.

© Yegor Bugayenko 2014–2018

XML+XSLT in a Browser

QR code

XML+XSLT in a Browser

Separating data and their presentation is a great concept. Take HTML and CSS for example. HTML is supposed to have pure data and CSS is supposed to format that data in order to make it readable by a human. Years ago, that was probably the intention of HTML/CSS, but in reality it doesn’t work like that. Mostly because CSS is not powerful enough.

We still have to format our data using HTML tags, while CSS can help slightly with positioning and decorating.

On the other hand, XML with XSLT implements perfectly the idea of separating data and presentation. XML documents, like HTML, are supposed to contain data only without any information about positioning or formatting. XSL stylesheets position and decorate the data. XSL is a much more powerful language. That’s why it’s possible to avoid any formatting inside XML.

The latest versions of Chrome, Safari, Firefox and IE all support this mechanism. When a browser retrieves an XML document from a server, and the document has an XSL stylesheet associated with it—the browser transforms XML into HTML on-fly.

Working Example

Let’s review a simple Java web application that works this way. It is using Takes Framework that makes this mechanism possible. In the next post, I’ll explain how ReXSL works. For now, though, let’s focus on the idea of delivering bare data in XML and formatting it with an XSL stylesheet.

Open http://www.stateful.co—it is a collection of stateful web primitives, explained in the Atomic Counters at Stateful.co article.

Open it in Chrome or Safari. When you do, you should see a normal web page with a logo, some text, some links, a footer, etc. Now check its sources (I assume you know how to do this).

This is approximately what you will see (I assume you understand XML, if not, start learning it immediately):

<?xml-stylesheet type='text/xsl' href='/xsl/index.xsl'?>
<page date="2014-06-15T15:30:49.521Z" ip="10.168.29.135">
  <menu>home</menu>
  <documentation>.. some text here ..</documentation>
  <version>
    <name>1.4</name>
    <revision>5c7b5af</revision>
    <date>2014-05-29 07:58</date>
  </version>
  <links>
    <link href="..." rel="rexsl:google" type="text/xml"/>
    <link href="..." rel="rexsl:github" type="text/xml"/>
    <link href="..." rel="rexsl:facebook" type="text/xml"/>
  </links>
  <millis>70</millis>
</page>

As you see, it is a proper XML document with attributes, elements and data. It contains absolutely no information about how its elements have to be presented to an end-user. Actually, this document is more suitable for machine parsing instead of reading by a human.

The document contains data, which is important for its requester. It’s up to the requester on how to render the data or to not render it at all.

Its second line associates the document with the XSL stylesheet /xsl/index.xsl that is loaded by the browser separately:

<?xml-stylesheet type='text/xsl' href='/xsl/index.xsl'?>

Open developer tools in Chrome and you will see that right after the page is loaded, the browser loads the XSL stylesheet and then all other resources including a few CSS stylesheets, jQuery and an SVG logo:

The figure

index.xsl includes layout.xsl, that’s why it is loaded right after.

Let’s consider an example of index.xsl (in reality it is much more complex, check layout.xsl. For example:

<xsl:stylesheet version="2.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns="http://www.w3.org/1999/xhtml">
  <xsl:template match="page">
    <html>
      <body>
        <p>
          Current version of the application is
          <xsl:value-of select="version/name"/>
        </p>
      </body>
    </html>
  </xsl:template>
</xsl:stylesheet>

I think it’s obvious how the HTML page will look like after applying this XSL stylesheet to our XML document.

For me, this XSL looks clean and easy to understand. However, I often hear people say that XSLT is a hard-to-understand programming language. I don’t find it hard to understand at all. Of course, I’m not using all of its features. But, for simple page rendering, all I need to know are a few simple commands and the principle of XML transformation.

Why Not a Templating Engine?

Now, why is this approach better than all that widely use Java templating engines, including JSP, JSF, Velocity, FreeMarker, Tiles, etc?

Well, I see a number of reasons. But, the most important are:

  1. Web UI and API are same pages. There is no need to develop separate pages for RESTful API—Web user interface, being accessed by a computer, is an API. In my experience, this leads to massive avoidance of code duplication.

  2. XSL is testable by itself without a server. In order to test how our web site will look with certain data, we just create a new XML document with necessary test data, associate it with an XSL and open it in a browser. We can also modify XML and refresh the page in browser. This makes the work of HTML/CSS designer much easier and independent of programmers.

  3. XSL is a powerful functional language. Compared with all other templating engines, which look mostly like workarounds, XSL is a complete and well-designed environment. Writing XSL (after you get used to its syntax and programming concepts) is a pleasure in itself. You’re not injecting instructions into a HTML document (like in JSP and all others). Instead, you are programming transformation of data into presentation—a different mindset and much better feeling.

  4. XML output is perfectly testable. A controller in MVC that generates an XML document with all data required for the XSL stylesheet can easily be tested in a single unit test using simple XPath expressions. Testing of a controller that injects data into a templating engine is a much more complex operation—even impossible sometimes. I’m also writing in PHP and Ruby. They have exactly the same problems—even though their templating engines are much more powerful due to the interpretation nature of the languages.

Is It Fully Supported?

Everything would be great if all browsers would support XML+XSL rendering. However, this is far from being true. Only the latest versions of modern browsers support XSL. Check this comparison done by Julian Reschke. Besides that, XSLT 2.0 is not supported at all.

There is a workaround, though. We can understand which browser is making a request (via its User-Agent HTTP header) and transform XML into HTML on the server side. Thus, for modern browsers that support XSL, we will deliver XML and for all others—HTML.

This is exactly how ReXSL framework works. Open http://www.stateful.co in Internet Explorer and you will see an HTML document, not an XML document as is the case with Chrome.

BTW, see how all this is implemented: XML Data and XSL Views in Takes Framework.

Read this one, it continues the discussion of this subject: RESTful API and a Web Site in the Same URL

Separating data and their presentation is a great concept. Take HTML and CSS for example. HTML is supposed to have pure data and CSS is supposed to format that data in order to make it readable by a human. Years ago, that was probably the intention of HTML/CSS, but in reality it doesn’t work like that. Mostly because CSS is not powerful enough.

We still have to format our data using HTML tags, while CSS can help slightly with positioning and decorating.

On the other hand, XML with XSLT implements perfectly the idea of separating data and presentation. XML documents, like HTML, are supposed to contain data only without any information about positioning or formatting. XSL stylesheets position and decorate the data. XSL is a much more powerful language. That’s why it’s possible to avoid any formatting inside XML.

The latest versions of Chrome, Safari, Firefox and IE all support this mechanism. When a browser retrieves an XML document from a server, and the document has an XSL stylesheet associated with it—the browser transforms XML into HTML on-fly.

Working Example

Let’s review a simple Java web application that works this way. It is using Takes Framework that makes this mechanism possible. In the next post, I’ll explain how ReXSL works. For now, though, let’s focus on the idea of delivering bare data in XML and formatting it with an XSL stylesheet.

Open http://www.stateful.co—it is a collection of stateful web primitives, explained in the Atomic Counters at Stateful.co article.

Open it in Chrome or Safari. When you do, you should see a normal web page with a logo, some text, some links, a footer, etc. Now check its sources (I assume you know how to do this).

This is approximately what you will see (I assume you understand XML, if not, start learning it immediately):

<?xml-stylesheet type='text/xsl' href='/xsl/index.xsl'?>
<page date="2014-06-15T15:30:49.521Z" ip="10.168.29.135">
  <menu>home</menu>
  <documentation>.. some text here ..</documentation>
  <version>
    <name>1.4</name>
    <revision>5c7b5af</revision>
    <date>2014-05-29 07:58</date>
  </version>
  <links>
    <link href="..." rel="rexsl:google" type="text/xml"/>
    <link href="..." rel="rexsl:github" type="text/xml"/>
    <link href="..." rel="rexsl:facebook" type="text/xml"/>
  </links>
  <millis>70</millis>
</page>

As you see, it is a proper XML document with attributes, elements and data. It contains absolutely no information about how its elements have to be presented to an end-user. Actually, this document is more suitable for machine parsing instead of reading by a human.

The document contains data, which is important for its requester. It’s up to the requester on how to render the data or to not render it at all.

Its second line associates the document with the XSL stylesheet /xsl/index.xsl that is loaded by the browser separately:

<?xml-stylesheet type='text/xsl' href='/xsl/index.xsl'?>

Open developer tools in Chrome and you will see that right after the page is loaded, the browser loads the XSL stylesheet and then all other resources including a few CSS stylesheets, jQuery and an SVG logo:

The figure

index.xsl includes layout.xsl, that’s why it is loaded right after.

Let’s consider an example of index.xsl (in reality it is much more complex, check layout.xsl. For example:

<xsl:stylesheet version="2.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns="http://www.w3.org/1999/xhtml">
  <xsl:template match="page">
    <html>
      <body>
        <p>
          Current version of the application is
          <xsl:value-of select="version/name"/>
        </p>
      </body>
    </html>
  </xsl:template>
</xsl:stylesheet>

I think it’s obvious how the HTML page will look like after applying this XSL stylesheet to our XML document.

For me, this XSL looks clean and easy to understand. However, I often hear people say that XSLT is a hard-to-understand programming language. I don’t find it hard to understand at all. Of course, I’m not using all of its features. But, for simple page rendering, all I need to know are a few simple commands and the principle of XML transformation.

Why Not a Templating Engine?

Now, why is this approach better than all that widely use Java templating engines, including JSP, JSF, Velocity, FreeMarker, Tiles, etc?

Well, I see a number of reasons. But, the most important are:

  1. Web UI and API are same pages. There is no need to develop separate pages for RESTful API—Web user interface, being accessed by a computer, is an API. In my experience, this leads to massive avoidance of code duplication.

  2. XSL is testable by itself without a server. In order to test how our web site will look with certain data, we just create a new XML document with necessary test data, associate it with an XSL and open it in a browser. We can also modify XML and refresh the page in browser. This makes the work of HTML/CSS designer much easier and independent of programmers.

  3. XSL is a powerful functional language. Compared with all other templating engines, which look mostly like workarounds, XSL is a complete and well-designed environment. Writing XSL (after you get used to its syntax and programming concepts) is a pleasure in itself. You’re not injecting instructions into a HTML document (like in JSP and all others). Instead, you are programming transformation of data into presentation—a different mindset and much better feeling.

  4. XML output is perfectly testable. A controller in MVC that generates an XML document with all data required for the XSL stylesheet can easily be tested in a single unit test using simple XPath expressions. Testing of a controller that injects data into a templating engine is a much more complex operation—even impossible sometimes. I’m also writing in PHP and Ruby. They have exactly the same problems—even though their templating engines are much more powerful due to the interpretation nature of the languages.

Is It Fully Supported?

Everything would be great if all browsers would support XML+XSL rendering. However, this is far from being true. Only the latest versions of modern browsers support XSL. Check this comparison done by Julian Reschke. Besides that, XSLT 2.0 is not supported at all.

There is a workaround, though. We can understand which browser is making a request (via its User-Agent HTTP header) and transform XML into HTML on the server side. Thus, for modern browsers that support XSL, we will deliver XML and for all others—HTML.

This is exactly how ReXSL framework works. Open http://www.stateful.co in Internet Explorer and you will see an HTML document, not an XML document as is the case with Chrome.

BTW, see how all this is implemented: XML Data and XSL Views in Takes Framework.

Read this one, it continues the discussion of this subject: RESTful API and a Web Site in the Same URL

© Yegor Bugayenko 2014–2018

Limit Java Method Execution Time

QR code

Limit Java Method Execution Time

badge

Say, you want to allow a Java method to work for a maximum of five seconds and want an exception to be thrown if the timeframe is exceeded. Here is how you can do it with jcabi-aspects and AspectJ:

public class Resource {
  @Timeable(limit = 5, unit = TimeUnit.SECONDS)
  public String load(URL url) {
    return url.openConnection().getContent();
  }
}

Keep in mind that you should weave your classes after compilation, as explained here.

Let’s discuss how this actually works, but first, I recommend you read this post, which explains how AOP aspects work together with Java annotations.

Due to @Timeable annotation and class weaving, every call to a method load() is intercepted by an aspect from jcabi-aspects. That aspect starts a new thread that monitors the execution of a method every second, checking whether it is still running.

If the method runs for over five seconds, the thread calls interrupt() on the method’s thread.

Despite a very common expectation that a thread should be terminated immediately on that call, it is not happening at all. This article explains the mechanism in more detail. Let’s discuss it briefly:

  1. interrupt() sets a marker in a thread;

  2. The thread checks interrupted() as often as it can;

  3. If the marker is set, the thread stops and throws InterruptedException

This method will not react to interrupt() call and will work until JVM is killed (very bad design):

public void work() {
  while (true) {
    // do something
  }
}

This is how we should refactor it in order to make sensitive to interruption requests:

public void work() {
  while (true) {
    if (Thread.interruped()) {
      throw new InterruptedException();
    }
    // do something
  }
}

In other words, your method can only stop itself. Nothing else can do it. The thread it is running in can’t be terminated by another thread. The best thing that the other thread can do is to send your thread a “message” (through interrupt() method) that it’s time to stop. If your thread ignores the message, nobody can do anything.

Most I/O operations in JDK are designed this way. They check the interruption status of their threads while waiting for I/O resources.

Thus, use @Timeable annotation, but keep in mind that there could be situations when a thread can’t be interrupted.

badge

Say, you want to allow a Java method to work for a maximum of five seconds and want an exception to be thrown if the timeframe is exceeded. Here is how you can do it with jcabi-aspects and AspectJ:

public class Resource {
  @Timeable(limit = 5, unit = TimeUnit.SECONDS)
  public String load(URL url) {
    return url.openConnection().getContent();
  }
}

Keep in mind that you should weave your classes after compilation, as explained here.

Let’s discuss how this actually works, but first, I recommend you read this post, which explains how AOP aspects work together with Java annotations.

Due to @Timeable annotation and class weaving, every call to a method load() is intercepted by an aspect from jcabi-aspects. That aspect starts a new thread that monitors the execution of a method every second, checking whether it is still running.

If the method runs for over five seconds, the thread calls interrupt() on the method’s thread.

Despite a very common expectation that a thread should be terminated immediately on that call, it is not happening at all. This article explains the mechanism in more detail. Let’s discuss it briefly:

  1. interrupt() sets a marker in a thread;

  2. The thread checks interrupted() as often as it can;

  3. If the marker is set, the thread stops and throws InterruptedException

This method will not react to interrupt() call and will work until JVM is killed (very bad design):

public void work() {
  while (true) {
    // do something
  }
}

This is how we should refactor it in order to make sensitive to interruption requests:

public void work() {
  while (true) {
    if (Thread.interruped()) {
      throw new InterruptedException();
    }
    // do something
  }
}

In other words, your method can only stop itself. Nothing else can do it. The thread it is running in can’t be terminated by another thread. The best thing that the other thread can do is to send your thread a “message” (through interrupt() method) that it’s time to stop. If your thread ignores the message, nobody can do anything.

Most I/O operations in JDK are designed this way. They check the interruption status of their threads while waiting for I/O resources.

Thus, use @Timeable annotation, but keep in mind that there could be situations when a thread can’t be interrupted.

© Yegor Bugayenko 2014–2018

Avoid String Concatenation

QR code

Avoid String Concatenation

This is “string concatenation,” and it is a bad practice:

// bad practice, don't reuse!
String text = "Hello, " + name + "!";

Why? Some may say that it is slow, mostly because parts of the resulting string are copied multiple times. Indeed, on every + operator, String class allocates a new block in memory and copies everything it has into it; plus a suffix being concatenated. This is true, but this is not the point here.

Actually, I don’t think performance in this case is a big issue. Moreover, there were multiple experiments showing that concatenation is not that slow when compared to other string building methods and sometimes is even faster.

Some say that concatenated strings are not localizable because in different languages text blocks in a phrase may be positioned in a different order. The example above can’t be translated to, say, Russian, where we would want to put a name in front of “привет.” We will need to localize the entire block of code, instead of just translating a phrase.

However, my point here is different. I strongly recommend avoiding string concatenation because it is less readable than other methods of joining texts together.

Let’s see these alternative methods. I’d recommend three of them (in order of preference): String.format(), Apache StringUtils and Guava Joiner.

There is also a StringBuilder, but I don’t find it as attractive as StringUtils. It is a useful builder of strings, but not a proper replacer or string concatenation tool when readability is important.

String.format()

String.format() is my favorite option. It makes text phrases easy to understand and modify. It is a static utility method that mirrors sprintf() from C. It allows you to build a string using a pattern and substitutors:

String text = String.format("Hello, %s!", name);

When the text is longer, the advantages of the formatter become much more obvious. Look at this ugly code:

String msg = "Dear " + customer.name()
  + ", your order #" + order.number()
  + " has been shipped at " + shipment.date()
  + "!";

This one looks much more beautiful doesn’t it:

String msg = String.format(
  "Dear %1$s, your order #%2$d has been shipped at %3$tR!",
  customer.name(), order.number(), shipment.date()
);

Please note that I’m using argument indexes in order to make the pattern even more localizable. Let’s say, I want to translate it to Greek. This is how will it look:

Αγαπητέ %1$s, στις %3$tR στείλαμε την παραγγελία σου με αριθμό #%2$d!

I’m changing the order of substitutions in the pattern, but not in the actual list of methods arguments.

Apache StringUtils.join()

When the text is rather long (longer than your screen width), I would recommend that you use the utility class StringUtils from Apache commons-lang3:

import org.apache.commons.lang3.StringUtils;
String xml = StringUtils.join(
  "<?xml version='1.0'?>",
  "<html><body>",
  "<p>This is a test XHTML document,",
  " which would look ugly,",
  " if we would use a single line,"
  " or string concatenation or String format().</p>"
  "</body></html>"
);

The need to include an additional JAR dependency to your classpath may be considered a downside with this method (get its latest versions in Maven Central):

<dependency>
  <groupId>org.apache.commons</groupId>
  <artifactId>commons-lang3</artifactId>
</dependency>

Guava Joiner

Similar functionality is provided by Joiner from Google Guava:

import com.google.common.base.Joiner;
String text = Joiner.on('').join(
  "WE HAVE BUNNY.\n",
  "GATHER ONE MILLION DOLLARS IN UNMARKED ",
  "NON-CONSECUTIVE TWENTIES.\n",
  "AWAIT INSTRUCTIONS.\n",
  "NO FUNNY STUFF"
);

It is a bit less convenient than StringUtils since you always have to provide a joiner (character or a string placed between text blocks).

Again, a dependency is required in this case:

<dependency>
  <groupId>com.google.guava</groupId>
  <artifactId>guava</artifactId>
</dependency>

Yes, in most cases, all of these methods work slower than a plain simple concatenation. However, I strongly believe that computers are cheaper than people. What I mean is that the time spent by programmers understanding and modifying ugly code is much more expensive than a cost of an additional server that will make beautifully written code work faster.

If you know any other methods of avoiding string concatenation, please comment below.

This is “string concatenation,” and it is a bad practice:

// bad practice, don't reuse!
String text = "Hello, " + name + "!";

Why? Some may say that it is slow, mostly because parts of the resulting string are copied multiple times. Indeed, on every + operator, String class allocates a new block in memory and copies everything it has into it; plus a suffix being concatenated. This is true, but this is not the point here.

Actually, I don’t think performance in this case is a big issue. Moreover, there were multiple experiments showing that concatenation is not that slow when compared to other string building methods and sometimes is even faster.

Some say that concatenated strings are not localizable because in different languages text blocks in a phrase may be positioned in a different order. The example above can’t be translated to, say, Russian, where we would want to put a name in front of “привет.” We will need to localize the entire block of code, instead of just translating a phrase.

However, my point here is different. I strongly recommend avoiding string concatenation because it is less readable than other methods of joining texts together.

Let’s see these alternative methods. I’d recommend three of them (in order of preference): String.format(), Apache StringUtils and Guava Joiner.

There is also a StringBuilder, but I don’t find it as attractive as StringUtils. It is a useful builder of strings, but not a proper replacer or string concatenation tool when readability is important.

String.format()

String.format() is my favorite option. It makes text phrases easy to understand and modify. It is a static utility method that mirrors sprintf() from C. It allows you to build a string using a pattern and substitutors:

String text = String.format("Hello, %s!", name);

When the text is longer, the advantages of the formatter become much more obvious. Look at this ugly code:

String msg = "Dear " + customer.name()
  + ", your order #" + order.number()
  + " has been shipped at " + shipment.date()
  + "!";

This one looks much more beautiful doesn’t it:

String msg = String.format(
  "Dear %1$s, your order #%2$d has been shipped at %3$tR!",
  customer.name(), order.number(), shipment.date()
);

Please note that I’m using argument indexes in order to make the pattern even more localizable. Let’s say, I want to translate it to Greek. This is how will it look:

Αγαπητέ %1$s, στις %3$tR στείλαμε την παραγγελία σου με αριθμό #%2$d!

I’m changing the order of substitutions in the pattern, but not in the actual list of methods arguments.

Apache StringUtils.join()

When the text is rather long (longer than your screen width), I would recommend that you use the utility class StringUtils from Apache commons-lang3:

import org.apache.commons.lang3.StringUtils;
String xml = StringUtils.join(
  "<?xml version='1.0'?>",
  "<html><body>",
  "<p>This is a test XHTML document,",
  " which would look ugly,",
  " if we would use a single line,"
  " or string concatenation or String format().</p>"
  "</body></html>"
);

The need to include an additional JAR dependency to your classpath may be considered a downside with this method (get its latest versions in Maven Central):

<dependency>
  <groupId>org.apache.commons</groupId>
  <artifactId>commons-lang3</artifactId>
</dependency>

Guava Joiner

Similar functionality is provided by Joiner from Google Guava:

import com.google.common.base.Joiner;
String text = Joiner.on('').join(
  "WE HAVE BUNNY.\n",
  "GATHER ONE MILLION DOLLARS IN UNMARKED ",
  "NON-CONSECUTIVE TWENTIES.\n",
  "AWAIT INSTRUCTIONS.\n",
  "NO FUNNY STUFF"
);

It is a bit less convenient than StringUtils since you always have to provide a joiner (character or a string placed between text blocks).

Again, a dependency is required in this case:

<dependency>
  <groupId>com.google.guava</groupId>
  <artifactId>guava</artifactId>
</dependency>

Yes, in most cases, all of these methods work slower than a plain simple concatenation. However, I strongly believe that computers are cheaper than people. What I mean is that the time spent by programmers understanding and modifying ugly code is much more expensive than a cost of an additional server that will make beautifully written code work faster.

If you know any other methods of avoiding string concatenation, please comment below.

© Yegor Bugayenko 2014–2018

Java Method Logging with AOP and Annotations

QR code

Java Method Logging with AOP and Annotations

Sometimes, I want to log (through slf4j and log4j) every execution of a method, seeing what arguments it receives, what it returns and how much time every execution takes. This is how I’m doing it, with help of AspectJ, jcabi-aspects and Java 6 annotations:

public class Foo {
  @Loggable
  public int power(int x, int p) {
    return Math.pow(x, p);
  }
}

This is what I see in log4j output:

[INFO] com.example.Foo #power(2, 10): 1024 in 12μs
[INFO] com.example.Foo #power(3, 3): 27 in 4μs

Nice, isn’t it? Now, let’s see how it works.

Annotation with Runtime Retention

Annotations is a technique introduced in Java 6. It is a meta-programming instrument that doesn’t change the way code works, but gives marks to certain elements (methods, classes or variables). In other words, annotations are just markers attached to the code that can be seen and read. Some annotations are designed to be seen at compile time only—they don’t exist in .class files after compilation. Others remain visible after compilation and can be accessed in runtime.

For example, @Override is of the first type (its retention type is SOURCE), while @Test from JUnit is of the second type (retention type is RUNTIME). @Loggable—the one I’m using in the script above—is an annotation of the second type, from jcabi-aspects. It stays with the byte-code in the .class file after compilation.

Again, it is important to understand that even though method power() is annotated and compiled, it doesn’t send anything to slf4j so far. It just contains a marker saying “please, log my execution.”

Aspect Oriented Programming (AOP)

AOP is a useful technique that enables adding executable blocks to the source code without explicitly changing it. In our example, we don’t want to log method execution inside the class. Instead, we want some other class to intercept every call to method power(), measure its execution time and send this information to slf4j.

We want that interceptor to understand our @Loggable annotation and log every call to that specific method power(). And, of course, the same interceptor should be used for other methods where we’ll place the same annotation in the future.

This case perfectly fits the original intent of AOP—to avoid re-implementation of some common behavior in multiple classes.

Logging is a supplementary feature to our main functionality, and we don’t want to pollute our code with multiple logging instructions. Instead, we want logging to happen behind the scenes.

In terms of AOP, our solution can be explained as creating an aspect that cross-cuts the code at certain join points and applies an around advice that implements the desired functionality.

AspectJ

Let’s see what these magic words mean. But, first, let’s see how jcabi-aspects implements them using AspectJ (it’s a simplified example, full code you can find in MethodLogger.java):

@Aspect
public class MethodLogger {
  @Around("execution(* *(..)) && @annotation(Loggable)")
  public Object around(ProceedingJoinPoint point) {
    long start = System.currentTimeMillis();
    Object result = point.proceed();
    Logger.info(
      "#%s(%s): %s in %[msec]s",
      MethodSignature.class.cast(point.getSignature()).getMethod().getName(),
      point.getArgs(),
      result,
      System.currentTimeMillis() - start
    );
    return result;
  }
}

This is an aspect with a single around advice around() inside. The aspect is annotated with @Aspect and advice is annotated with @Around. As discussed above, these annotations are just markers in .class files. They don’t do anything except provide some meta-information to those who are interested in runtime.

Annotation @Around has one parameter, which—in this case—says that the advice should be applied to a method if:

  1. its visibility modifier is * (public, protected or private);

  2. its name is name * (any name);

  3. its arguments are .. (any arguments); and

  4. it is annotated with @Loggable

When a call to an annotated method is to be intercepted, method around() executes before executing the actual method. When a call to method power() is to be intercepted, method around() receives an instance of class ProceedingJoinPoint and must return an object, which will be used as a result of method power().

In order to call the original method, power(), the advice has to call proceed() of the join point object.

We compile this aspect and make it available in classpath together with our main file Foo.class. So far so good, but we need to take one last step in order to put our aspect into action—we should apply our advice.

Binary Aspect Weaving

Aspect weaving is the name of the advice applying process. Aspect weaver modifies original code by injecting calls to aspects. AspectJ does exactly that. We give it two binary Java classes Foo.class and MethodLogger.class; it gives back three—modified Foo.class, Foo$AjcClosure1.class and unmodified MethodLogger.class.

In order to understand which advice should be applied to which methods, AspectJ weaver is using annotations from .class files. Also, it uses reflection to browse all classes on classpath. It analyzes which methods satisfy the conditions from the @Around annotation. Of course, it finds our method power().

So, there are two steps. First, we compile our .java files using javac and get two files. Then, AspectJ weaves/modifies them and creates its own extra class. Our Foo class looks something like this after weaving:

public class Foo {
  private final MethodLogger logger;
  @Loggable
  public int power(int x, int p) {
    return this.logger.around(point);
  }
  private int power_aroundBody(int x, int p) {
    return Math.pow(x, p);
  }
}

AspectJ weaver moves our original functionality to a new method, power_aroundBody(), and redirects all power() calls to the aspect class MethodLogger.

Instead of one method power() in class Foo now we have four classes working together. From now on, this is what happens behind the scenes on every call to power():

PlantUML SVG diagram

Original functionality of method power() is indicated by the small green lifeline on the diagram.

As you see, the aspect weaving process connects together classes and aspects, transferring calls between them through join points. Without weaving, both classes and aspects are just compiled Java binaries with attached annotations.

jcabi-aspects

jcabi-aspects is a JAR library that contains Loggable annotation and MethodLogger aspect (btw, there are many more aspects and annotations). You don’t need to write your own aspect for method logging. Just add a few dependencies to your classpath and configure jcabi-maven-plugin for aspect weaving (get their latest versions in Maven Central):

<project>
  <dependencies>
    <dependency>
      <groupId>com.jcabi</groupId>
      <artifactId>jcabi-aspects</artifactId>
    </dependency>
    <dependency>
      <groupId>org.aspectj</groupId>
      <artifactId>aspectjrt</artifactId>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <groupId>com.jcabi</groupId>
        <artifactId>jcabi-maven-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>ajc</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Since this weaving procedure takes a lot of configuration effort, I created a convenient Maven plugin with an ajc goal, which does the entire aspect weaving job. You can use AspectJ directly, but I recommend that you use jcabi-maven-plugin.

That’s it. Now you can use @com.jcabi.aspects.Loggable annotation and your methods will be logged through slf4j.

If something doesn’t work as explained, don’t hesitate to submit a GitHub issue.

Sometimes, I want to log (through slf4j and log4j) every execution of a method, seeing what arguments it receives, what it returns and how much time every execution takes. This is how I’m doing it, with help of AspectJ, jcabi-aspects and Java 6 annotations:

public class Foo {
  @Loggable
  public int power(int x, int p) {
    return Math.pow(x, p);
  }
}

This is what I see in log4j output:

[INFO] com.example.Foo #power(2, 10): 1024 in 12μs
[INFO] com.example.Foo #power(3, 3): 27 in 4μs

Nice, isn’t it? Now, let’s see how it works.

Annotation with Runtime Retention

Annotations is a technique introduced in Java 6. It is a meta-programming instrument that doesn’t change the way code works, but gives marks to certain elements (methods, classes or variables). In other words, annotations are just markers attached to the code that can be seen and read. Some annotations are designed to be seen at compile time only—they don’t exist in .class files after compilation. Others remain visible after compilation and can be accessed in runtime.

For example, @Override is of the first type (its retention type is SOURCE), while @Test from JUnit is of the second type (retention type is RUNTIME). @Loggable—the one I’m using in the script above—is an annotation of the second type, from jcabi-aspects. It stays with the byte-code in the .class file after compilation.

Again, it is important to understand that even though method power() is annotated and compiled, it doesn’t send anything to slf4j so far. It just contains a marker saying “please, log my execution.”

Aspect Oriented Programming (AOP)

AOP is a useful technique that enables adding executable blocks to the source code without explicitly changing it. In our example, we don’t want to log method execution inside the class. Instead, we want some other class to intercept every call to method power(), measure its execution time and send this information to slf4j.

We want that interceptor to understand our @Loggable annotation and log every call to that specific method power(). And, of course, the same interceptor should be used for other methods where we’ll place the same annotation in the future.

This case perfectly fits the original intent of AOP—to avoid re-implementation of some common behavior in multiple classes.

Logging is a supplementary feature to our main functionality, and we don’t want to pollute our code with multiple logging instructions. Instead, we want logging to happen behind the scenes.

In terms of AOP, our solution can be explained as creating an aspect that cross-cuts the code at certain join points and applies an around advice that implements the desired functionality.

AspectJ

Let’s see what these magic words mean. But, first, let’s see how jcabi-aspects implements them using AspectJ (it’s a simplified example, full code you can find in MethodLogger.java):

@Aspect
public class MethodLogger {
  @Around("execution(* *(..)) && @annotation(Loggable)")
  public Object around(ProceedingJoinPoint point) {
    long start = System.currentTimeMillis();
    Object result = point.proceed();
    Logger.info(
      "#%s(%s): %s in %[msec]s",
      MethodSignature.class.cast(point.getSignature()).getMethod().getName(),
      point.getArgs(),
      result,
      System.currentTimeMillis() - start
    );
    return result;
  }
}

This is an aspect with a single around advice around() inside. The aspect is annotated with @Aspect and advice is annotated with @Around. As discussed above, these annotations are just markers in .class files. They don’t do anything except provide some meta-information to those who are interested in runtime.

Annotation @Around has one parameter, which—in this case—says that the advice should be applied to a method if:

  1. its visibility modifier is * (public, protected or private);

  2. its name is name * (any name);

  3. its arguments are .. (any arguments); and

  4. it is annotated with @Loggable

When a call to an annotated method is to be intercepted, method around() executes before executing the actual method. When a call to method power() is to be intercepted, method around() receives an instance of class ProceedingJoinPoint and must return an object, which will be used as a result of method power().

In order to call the original method, power(), the advice has to call proceed() of the join point object.

We compile this aspect and make it available in classpath together with our main file Foo.class. So far so good, but we need to take one last step in order to put our aspect into action—we should apply our advice.

Binary Aspect Weaving

Aspect weaving is the name of the advice applying process. Aspect weaver modifies original code by injecting calls to aspects. AspectJ does exactly that. We give it two binary Java classes Foo.class and MethodLogger.class; it gives back three—modified Foo.class, Foo$AjcClosure1.class and unmodified MethodLogger.class.

In order to understand which advice should be applied to which methods, AspectJ weaver is using annotations from .class files. Also, it uses reflection to browse all classes on classpath. It analyzes which methods satisfy the conditions from the @Around annotation. Of course, it finds our method power().

So, there are two steps. First, we compile our .java files using javac and get two files. Then, AspectJ weaves/modifies them and creates its own extra class. Our Foo class looks something like this after weaving:

public class Foo {
  private final MethodLogger logger;
  @Loggable
  public int power(int x, int p) {
    return this.logger.around(point);
  }
  private int power_aroundBody(int x, int p) {
    return Math.pow(x, p);
  }
}

AspectJ weaver moves our original functionality to a new method, power_aroundBody(), and redirects all power() calls to the aspect class MethodLogger.

Instead of one method power() in class Foo now we have four classes working together. From now on, this is what happens behind the scenes on every call to power():

PlantUML SVG diagram

Original functionality of method power() is indicated by the small green lifeline on the diagram.

As you see, the aspect weaving process connects together classes and aspects, transferring calls between them through join points. Without weaving, both classes and aspects are just compiled Java binaries with attached annotations.

jcabi-aspects

jcabi-aspects is a JAR library that contains Loggable annotation and MethodLogger aspect (btw, there are many more aspects and annotations). You don’t need to write your own aspect for method logging. Just add a few dependencies to your classpath and configure jcabi-maven-plugin for aspect weaving (get their latest versions in Maven Central):

<project>
  <dependencies>
    <dependency>
      <groupId>com.jcabi</groupId>
      <artifactId>jcabi-aspects</artifactId>
    </dependency>
    <dependency>
      <groupId>org.aspectj</groupId>
      <artifactId>aspectjrt</artifactId>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <groupId>com.jcabi</groupId>
        <artifactId>jcabi-maven-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>ajc</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Since this weaving procedure takes a lot of configuration effort, I created a convenient Maven plugin with an ajc goal, which does the entire aspect weaving job. You can use AspectJ directly, but I recommend that you use jcabi-maven-plugin.

That’s it. Now you can use @com.jcabi.aspects.Loggable annotation and your methods will be logged through slf4j.

If something doesn’t work as explained, don’t hesitate to submit a GitHub issue.

© Yegor Bugayenko 2014–2018

Object-Oriented Java Adapter of Amazon S3 SDK

QR code

Object-Oriented Java Adapter of Amazon S3 SDK

badge

I’m a big fan of Amazon Web Services (AWS). I’m using them in almost all of my projects. One of their most popular services is Simple Storage Service (S3). It is a storage for binary objects (files) with unique names, accessible through HTTP or RESTful API.

Using S3 is very simple. You create a “bucket” with a unique name, upload your “object” into the bucket through their web interface or through RESTful API, and then download it again (either through HTTP or the API.)

Amazon ships the Java SDK that wraps their RESTful API. However, this SDK is not object-oriented at all. It is purely imperative and procedural—it just mirrors the API.

For example, in order to download an existing object doc.txt from bucket test-1, you have to do something like this:

AWSCredentials creds = new BasicAWSCredentials(key, secret);
AmazonS3 aws = new AmazonS3Client(creds);
S3Object obj = aws.getObject(
  new GetObjectRequest("test-1", "doc.txt")
);
InputStream input = obj.getObjectContent();
String content = IOUtils.toString(input, "UTF-8");
input.close();
badge

As always, procedural programming has its inevitable disadvantages. To overcome them all, I designed jcabi-s3, which is a small object-oriented adapter for Amazon SDK. This is how the same object-reading task can be accomplished with jcabi-s3:

Region region = new Region.Simple(key, secret);
Bucket bucket = region.bucket("test-1");
Ocket ocket = bucket.ocket("doc.txt");
String content = new Ocket.Text(ocket).read();

Why is this approach better? Well, there are a number of obvious advantages.

S3 Object is an Object in Java

S3 object get its representative in Java. It is not a collection of procedures to be called in order to get its properties (as with AWS SDK). Rather, it is a Java object with certain behaviors. I called them “ockets” (similar to “buckets”), in order to avoid clashes with java.lang.Object.

Ocket is an interface, that exposes the behavior of a real AWS S3 object: read, write, check existence. There is also a convenient decorator Ocket.Text that simplifies working with binary objects:

Ocket.Text ocket = new Ocket.Text(ocket_from_s3);
if (ocket.exists()) {
  System.out.print(ocket.read());
} else {
  ocket.write("Hello, world!");
}

Now, you can pass an object to another class, instead of giving it your AWS credentials, bucket name, and object name. You simply pass a Java object, which encapsulates all AWS interaction details.

Extendability Through Decoration

Since jcabi-s3 exposes all entities as interfaces, they can easily be extended through encapsulation (Decorator Pattern).

For example, you want your code to retry S3 object read operations a few times before giving up and throwing an IOException (by the way, this is a very good practice when working with web services). So, you want all your S3 reading operations to be redone a few times if first attempts fail.

You define a new decorator class, say, RetryingOcket, which encapsulates an original Ocket:

public RetryingOcket implements Ocket {
  private final Ocket origin;
  public RetryingOcket(Ocket ocket) {
    this.origin = ocket;
  }
  @Override
  public void read(OutputStream stream) throws IOException {
    int attempt = 0;
    while (true) {
      try {
        this.origin.read(stream);
      } catch (IOException ex) {
        if (attempt++ > 3) {
          throw ex;
        }
      }
    }
  }
  // same for other methods
}

Now, everywhere where Ocket is expected you send an instance of RetryingOcket that wraps your original object:

foo.process(new RetryingOcket(ocket));

Method foo.process() won’t see a difference, since it is the same Ocket interface it is expecting.

By the way, this retry functionality is implemented out-of-the-box in jcabi-s3, in com.jcabi.s3.retry package.

Easy Mocking

Again, due to the fact that all entities in jcabi-s3 are interfaces, they are very easy to mock. For example, your class expects an S3 object, reads its data and calculates the MD5 hash (I’m using DigestUtils from commons-codec):

import com.jcabi.s3.Ocket;
import org.apache.commons.codec.digest.DigestUtils;
public class S3Md5Hash {
  private final Ocket ocket;
  public S3Md5Hash(Ocket okt) {
    this.ocket = okt;
  }
  public hash() throws IOException {
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    this.ocket.read(baos);
    return DigestUtils.md5hex(baos.toByteArray());
  }
}

Here is how simple a unit test will look (try to create a unit test for a class using AWS SDK and you will see the difference):

import com.jcabi.s3.Ocket;
import org.junit.Test;
public class S3Md5HashTest {
  @Test
  public void generatesHash() {
    Ocket ocket = Mockito.mock(Ocket.class);
    Mockito.doAnswer(
      new Answer<Void>() {
        public Void answer(final InvocationOnMock inv) throws IOException {
          OutputStream.class.cast(inv.getArguments()[0]).write(' ');
        }
      }
    ).when(ocket).read(Mockito.any(OutputStream.class));
    String hash = new S5Md5Hash(ocket);
    Assert.assertEquals(hash, "7215ee9c7d9dc229d2921a40e899ec5f");
  }
}

I’m using JUnit and Mockito in this test.

Immutability

All classes in jcabi-s3 are annotated with @Immutable and are truly immutable.

The library ships as a JAR dependency in Maven Central (get its latest versions in Maven Central):

<dependency>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-s3</artifactId>
</dependency>

As always, your comments and criticism are welcome as GitHub issues.

badge

I’m a big fan of Amazon Web Services (AWS). I’m using them in almost all of my projects. One of their most popular services is Simple Storage Service (S3). It is a storage for binary objects (files) with unique names, accessible through HTTP or RESTful API.

Using S3 is very simple. You create a “bucket” with a unique name, upload your “object” into the bucket through their web interface or through RESTful API, and then download it again (either through HTTP or the API.)

Amazon ships the Java SDK that wraps their RESTful API. However, this SDK is not object-oriented at all. It is purely imperative and procedural—it just mirrors the API.

For example, in order to download an existing object doc.txt from bucket test-1, you have to do something like this:

AWSCredentials creds = new BasicAWSCredentials(key, secret);
AmazonS3 aws = new AmazonS3Client(creds);
S3Object obj = aws.getObject(
  new GetObjectRequest("test-1", "doc.txt")
);
InputStream input = obj.getObjectContent();
String content = IOUtils.toString(input, "UTF-8");
input.close();
badge

As always, procedural programming has its inevitable disadvantages. To overcome them all, I designed jcabi-s3, which is a small object-oriented adapter for Amazon SDK. This is how the same object-reading task can be accomplished with jcabi-s3:

Region region = new Region.Simple(key, secret);
Bucket bucket = region.bucket("test-1");
Ocket ocket = bucket.ocket("doc.txt");
String content = new Ocket.Text(ocket).read();

Why is this approach better? Well, there are a number of obvious advantages.

S3 Object is an Object in Java

S3 object get its representative in Java. It is not a collection of procedures to be called in order to get its properties (as with AWS SDK). Rather, it is a Java object with certain behaviors. I called them “ockets” (similar to “buckets”), in order to avoid clashes with java.lang.Object.

Ocket is an interface, that exposes the behavior of a real AWS S3 object: read, write, check existence. There is also a convenient decorator Ocket.Text that simplifies working with binary objects:

Ocket.Text ocket = new Ocket.Text(ocket_from_s3);
if (ocket.exists()) {
  System.out.print(ocket.read());
} else {
  ocket.write("Hello, world!");
}

Now, you can pass an object to another class, instead of giving it your AWS credentials, bucket name, and object name. You simply pass a Java object, which encapsulates all AWS interaction details.

Extendability Through Decoration

Since jcabi-s3 exposes all entities as interfaces, they can easily be extended through encapsulation (Decorator Pattern).

For example, you want your code to retry S3 object read operations a few times before giving up and throwing an IOException (by the way, this is a very good practice when working with web services). So, you want all your S3 reading operations to be redone a few times if first attempts fail.

You define a new decorator class, say, RetryingOcket, which encapsulates an original Ocket:

public RetryingOcket implements Ocket {
  private final Ocket origin;
  public RetryingOcket(Ocket ocket) {
    this.origin = ocket;
  }
  @Override
  public void read(OutputStream stream) throws IOException {
    int attempt = 0;
    while (true) {
      try {
        this.origin.read(stream);
      } catch (IOException ex) {
        if (attempt++ > 3) {
          throw ex;
        }
      }
    }
  }
  // same for other methods
}

Now, everywhere where Ocket is expected you send an instance of RetryingOcket that wraps your original object:

foo.process(new RetryingOcket(ocket));

Method foo.process() won’t see a difference, since it is the same Ocket interface it is expecting.

By the way, this retry functionality is implemented out-of-the-box in jcabi-s3, in com.jcabi.s3.retry package.

Easy Mocking

Again, due to the fact that all entities in jcabi-s3 are interfaces, they are very easy to mock. For example, your class expects an S3 object, reads its data and calculates the MD5 hash (I’m using DigestUtils from commons-codec):

import com.jcabi.s3.Ocket;
import org.apache.commons.codec.digest.DigestUtils;
public class S3Md5Hash {
  private final Ocket ocket;
  public S3Md5Hash(Ocket okt) {
    this.ocket = okt;
  }
  public hash() throws IOException {
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    this.ocket.read(baos);
    return DigestUtils.md5hex(baos.toByteArray());
  }
}

Here is how simple a unit test will look (try to create a unit test for a class using AWS SDK and you will see the difference):

import com.jcabi.s3.Ocket;
import org.junit.Test;
public class S3Md5HashTest {
  @Test
  public void generatesHash() {
    Ocket ocket = Mockito.mock(Ocket.class);
    Mockito.doAnswer(
      new Answer<Void>() {
        public Void answer(final InvocationOnMock inv) throws IOException {
          OutputStream.class.cast(inv.getArguments()[0]).write(' ');
        }
      }
    ).when(ocket).read(Mockito.any(OutputStream.class));
    String hash = new S5Md5Hash(ocket);
    Assert.assertEquals(hash, "7215ee9c7d9dc229d2921a40e899ec5f");
  }
}

I’m using JUnit and Mockito in this test.

Immutability

All classes in jcabi-s3 are annotated with @Immutable and are truly immutable.

The library ships as a JAR dependency in Maven Central (get its latest versions in Maven Central):

<dependency>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-s3</artifactId>
</dependency>

As always, your comments and criticism are welcome as GitHub issues.

© Yegor Bugayenko 2014–2018

Get Rid of Java Static Loggers

QR code

Get Rid of Java Static Loggers

This is a very common practice in Java (using LoggerFactory from slf4j):

import org.slf4j.LoggerFactory;
public class Foo {
  private static final Logger LOGGER =
    LoggerFactory.getLogger(Foo.class);
  public void save(String file) {
    // save the file
    if (Foo.LOGGER.isInfoEnabled()) {
      Foo.LOGGER.info("file {} saved successfuly", file);
    }
  }
}

What’s wrong with it? Code duplication.

This static LOGGER property has to be declared in every class where logging is required. Just a few lines of code, but this is pure noise, as I see it.

badge

To make life easier, I created a library about two years ago, jcabi-log, which has a convenient utility class Logger (yes, I know that utility classes are evil).

import com.jcabi.log.Logger;
public class Foo {
  public void save(String file) {
    // save the file
    Logger.info(this, "file %s saved successfuly", file);
  }
}

This looks much cleaner to me and does exactly the same—sends a single log line to the SLF4J logging facility. Besides, it check automatically whether a given logging level is enabled (for performance optimization) and formats the given string using Formatter (same as String.format()).

For convenience, there are also a number of “decors” implemented in the library.

The library ships as a JAR dependency in Maven Central (get its latest versions in Maven Central):

<dependency>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-log</artifactId>
</dependency>

This is a very common practice in Java (using LoggerFactory from slf4j):

import org.slf4j.LoggerFactory;
public class Foo {
  private static final Logger LOGGER =
    LoggerFactory.getLogger(Foo.class);
  public void save(String file) {
    // save the file
    if (Foo.LOGGER.isInfoEnabled()) {
      Foo.LOGGER.info("file {} saved successfuly", file);
    }
  }
}

What’s wrong with it? Code duplication.

This static LOGGER property has to be declared in every class where logging is required. Just a few lines of code, but this is pure noise, as I see it.

badge

To make life easier, I created a library about two years ago, jcabi-log, which has a convenient utility class Logger (yes, I know that utility classes are evil).

import com.jcabi.log.Logger;
public class Foo {
  public void save(String file) {
    // save the file
    Logger.info(this, "file %s saved successfuly", file);
  }
}

This looks much cleaner to me and does exactly the same—sends a single log line to the SLF4J logging facility. Besides, it check automatically whether a given logging level is enabled (for performance optimization) and formats the given string using Formatter (same as String.format()).

For convenience, there are also a number of “decors” implemented in the library.

The library ships as a JAR dependency in Maven Central (get its latest versions in Maven Central):

<dependency>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-log</artifactId>
</dependency>

© Yegor Bugayenko 2014–2018

MySQL Maven Plugin

QR code

MySQL Maven Plugin

I was using MySQL in a few Java web projects and found out there was no Maven plugin that would help me to test my DAOs against a real MySQL server. There are plenty of mechanisms to mock a database persistence layer both in memory and on disc. However, it is always good to make sure that your classes are tested against a database identical to the one you have in production environment.

badge

I’ve created my own Maven plugin, jcabi-mysql-maven-plugin, that does exactly two things: starts a MySQL server on pre-integration-test phase and shuts it down on post-integration-test.

This is how you configure it in pom.xml (see also its full usage instructions):

<project>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>reserve-network-port</goal>
            </goals>
            <configuration>
              <portNames>
                <portName>mysql.port</portName>
              </portNames>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>unpack</goal>
            </goals>
            <configuration>
              <artifactItems>
                <artifactItem>
                  <groupId>com.jcabi</groupId>
                  <artifactId>mysql-dist</artifactId>
                  <version>5.6.14</version>
                  <classifier>${mysql.classifier}</classifier>
                  <type>zip</type>
                  <overWrite>false</overWrite>
                  <outputDirectory>
                    ${project.build.directory}/mysql-dist
                  </outputDirectory>
                </artifactItem>
              </artifactItems>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>com.jcabi</groupId>
        <artifactId>jcabi-mysql-maven-plugin</artifactId>
        <executions>
          <execution>
            <id>mysql-test</id>
            <goals>
              <goal>classify</goal>
              <goal>start</goal>
              <goal>stop</goal>
            </goals>
            <configuration>
              <port>${mysql.port}</port>
              <data>${project.build.directory}/mysql-data</data>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <artifactId>maven-failsafe-plugin</artifactId>
        <configuration>
          <systemPropertyVariables>
            <mysql.port>${mysql.port}</mysql.port>
          </systemPropertyVariables>
        </configuration>
        <executions>
          <execution>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  [...]
</project>

There are two plugins configured above. Let’s take a look at what each does.

  1. build-helper-maven-plugin is reserving a temporary random TCP port, which will be used by MySQL server. We don’t want to start a server on its default 3306 port, because there could be another server already running there. Besides that, if we use a hard-coded TCP port, we won’t be able to run multiple builds in parallel. Maybe not a big deal when you’re developing locally, but in continuous integration environment this can be a problem. That’s why we’re reserving a TCP port first.

  2. maven-dependency-plugin is downloading a MySQL distribution in a zip archive (rather big file, over 300Mb for Linux), and unpacks it. This archive contains exactly the same files as you would use for a traditional MySQL installation. When the archive is unpacked, it is ready to start serving SQL requests as a normal MySQL server.

  3. jcabi-mysql-maven-plugin starts a server, binding it to a TCP port reserved randomly. The main responsibility of my Maven plugin is to make sure that MySQL server starts correctly on every platform (Mac OS, Linux, Windows) and stops when it’s not needed any more. All the rest is done by the MySQL distribution itself.

  4. maven-failsafe-plugin is running unit tests on integration-test phase. Its main difference from maven-surefire-plugin is that it doesn’t fail a build when some tests fail. Instead, it saves all failures into supplementary files in target directory and allows the build continue. Later, when we call its verify goal, it will fail a build if there were any errors during its integration-test goal execution.

To be precise, this is the order in which Maven will execute configured goals:

jcabi-mysql-maven-plugin:classify
maven-dependency-plugin:unpack
build-helper-maven-plugin:reserve-network-port
jcabi-mysql-maven-plugin:start
maven-failsafe-plugin:integration-test
jcabi-mysql-maven-plugin:stop
maven-failsafe-plugin:verify

Run mvn clean install and see how it works. If it doesn’t work for some reason, don’t hesitate to report an issue to GitHub.

Now it’s time to create an integration test, which will connect to the temporary MySQL server, create a table there and insert some data into it. This is just an example to show that MySQL server is running and is capable of serving transactions (I’m using jcabi-jdbc):

public class FooITCase {
  private static final String PORT = System.getProperty("mysql.port");
  @Test
  public void worksWithMysqlServer() {
    Connection conn = DriverManager.getConnection(
      String.format(
        "jdbc:mysql://localhost:%s/root?user=root&password=root",
        FooITCase.PORT
      )
    );
    new JdbcSession(conn)
      .sql("CREATE TABLE foo (id INT PRIMARY KEY)")
      .execute();
  }
}

If you’re using Hibernate, just create a db.properties file in src/test/resources directory. In that file you would do something like:

hibernate.connection.url=jdbc:mysql://localhost:${mysql.port}/root
hibernate.connection.username=root
hibernate.connection.password=root

Maven will replace that ${mysql.port} with the number of reserved TCP port, during resources copying. This operation is called “resources filtering,” and you can read about it here.

That’s pretty much it. I’m using jcabi-mysql-maven-plugin in a few projects, and it helps me to stay confident that my code works with a real MySQL server. I’m also using the Liquibase Maven plugin in order to populate an empty server with tables required for the application. Nevertheless, that is a story for the next post :)

I was using MySQL in a few Java web projects and found out there was no Maven plugin that would help me to test my DAOs against a real MySQL server. There are plenty of mechanisms to mock a database persistence layer both in memory and on disc. However, it is always good to make sure that your classes are tested against a database identical to the one you have in production environment.

badge

I’ve created my own Maven plugin, jcabi-mysql-maven-plugin, that does exactly two things: starts a MySQL server on pre-integration-test phase and shuts it down on post-integration-test.

This is how you configure it in pom.xml (see also its full usage instructions):

<project>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>reserve-network-port</goal>
            </goals>
            <configuration>
              <portNames>
                <portName>mysql.port</portName>
              </portNames>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>unpack</goal>
            </goals>
            <configuration>
              <artifactItems>
                <artifactItem>
                  <groupId>com.jcabi</groupId>
                  <artifactId>mysql-dist</artifactId>
                  <version>5.6.14</version>
                  <classifier>${mysql.classifier}</classifier>
                  <type>zip</type>
                  <overWrite>false</overWrite>
                  <outputDirectory>
                    ${project.build.directory}/mysql-dist
                  </outputDirectory>
                </artifactItem>
              </artifactItems>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>com.jcabi</groupId>
        <artifactId>jcabi-mysql-maven-plugin</artifactId>
        <executions>
          <execution>
            <id>mysql-test</id>
            <goals>
              <goal>classify</goal>
              <goal>start</goal>
              <goal>stop</goal>
            </goals>
            <configuration>
              <port>${mysql.port}</port>
              <data>${project.build.directory}/mysql-data</data>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <artifactId>maven-failsafe-plugin</artifactId>
        <configuration>
          <systemPropertyVariables>
            <mysql.port>${mysql.port}</mysql.port>
          </systemPropertyVariables>
        </configuration>
        <executions>
          <execution>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  [...]
</project>

There are two plugins configured above. Let’s take a look at what each does.

  1. build-helper-maven-plugin is reserving a temporary random TCP port, which will be used by MySQL server. We don’t want to start a server on its default 3306 port, because there could be another server already running there. Besides that, if we use a hard-coded TCP port, we won’t be able to run multiple builds in parallel. Maybe not a big deal when you’re developing locally, but in continuous integration environment this can be a problem. That’s why we’re reserving a TCP port first.

  2. maven-dependency-plugin is downloading a MySQL distribution in a zip archive (rather big file, over 300Mb for Linux), and unpacks it. This archive contains exactly the same files as you would use for a traditional MySQL installation. When the archive is unpacked, it is ready to start serving SQL requests as a normal MySQL server.

  3. jcabi-mysql-maven-plugin starts a server, binding it to a TCP port reserved randomly. The main responsibility of my Maven plugin is to make sure that MySQL server starts correctly on every platform (Mac OS, Linux, Windows) and stops when it’s not needed any more. All the rest is done by the MySQL distribution itself.

  4. maven-failsafe-plugin is running unit tests on integration-test phase. Its main difference from maven-surefire-plugin is that it doesn’t fail a build when some tests fail. Instead, it saves all failures into supplementary files in target directory and allows the build continue. Later, when we call its verify goal, it will fail a build if there were any errors during its integration-test goal execution.

To be precise, this is the order in which Maven will execute configured goals:

jcabi-mysql-maven-plugin:classify
maven-dependency-plugin:unpack
build-helper-maven-plugin:reserve-network-port
jcabi-mysql-maven-plugin:start
maven-failsafe-plugin:integration-test
jcabi-mysql-maven-plugin:stop
maven-failsafe-plugin:verify

Run mvn clean install and see how it works. If it doesn’t work for some reason, don’t hesitate to report an issue to GitHub.

Now it’s time to create an integration test, which will connect to the temporary MySQL server, create a table there and insert some data into it. This is just an example to show that MySQL server is running and is capable of serving transactions (I’m using jcabi-jdbc):

public class FooITCase {
  private static final String PORT = System.getProperty("mysql.port");
  @Test
  public void worksWithMysqlServer() {
    Connection conn = DriverManager.getConnection(
      String.format(
        "jdbc:mysql://localhost:%s/root?user=root&password=root",
        FooITCase.PORT
      )
    );
    new JdbcSession(conn)
      .sql("CREATE TABLE foo (id INT PRIMARY KEY)")
      .execute();
  }
}

If you’re using Hibernate, just create a db.properties file in src/test/resources directory. In that file you would do something like:

hibernate.connection.url=jdbc:mysql://localhost:${mysql.port}/root
hibernate.connection.username=root
hibernate.connection.password=root

Maven will replace that ${mysql.port} with the number of reserved TCP port, during resources copying. This operation is called “resources filtering,” and you can read about it here.

That’s pretty much it. I’m using jcabi-mysql-maven-plugin in a few projects, and it helps me to stay confident that my code works with a real MySQL server. I’m also using the Liquibase Maven plugin in order to populate an empty server with tables required for the application. Nevertheless, that is a story for the next post :)

© Yegor Bugayenko 2014–2018

DynamoDB Local Maven Plugin

QR code

DynamoDB Local Maven Plugin

badge

DynamoDB Local is a locally running copy of Amazon DynamoDB server. Amazon developed the tool and based it on SQLite. It acts as a real DynamoDB service through the RESTful API.

I guess, DynamoDB Local is meant to be used in integration testing and this is how we’re going to use it below.

I use Maven to run all of my Java integration testing using maven-failsafe-plugin. The philosophy of integration testing with Maven is that you start all your supplementary test stubs during the pre-integration-test phase, run your tests in the integration-test phase and then shutdown all stubs during the post-integration-test.

badge

It would be great if it were possible to use DynamoDB Local that way. I didn’t find any Maven plugins for that purpose, so I decided to create my own—jcabi-dynamodb-maven-plugin.

Full usage details for the plugin are explained on its website. However, here is a simple example (get its latest versions in Maven Central):

<plugin>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-dynamodb-maven-plugin</artifactId>
  <executions>
    <execution>
      <goals>
        <goal>start</goal>
        <goal>stop</goal>
      </goals>
      <configuration>
        <port>10500</port>
        <dist>${project.build.directory}/dynamodb-dist</dist>
      </configuration>
    </execution>
  </executions>
</plugin>

The above configuration will start DynamoDB Local right before running integration tests, and then stop it immediately afterwards. The server will listen at TCP port 10500. While the number is used in the example, you’re supposed to use a randomly allocated port instead.

When the DynamoDB Local server is up and running, we can create an integration test for it:

import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.model.ListTablesResult;
public class FooITCase {
  @Test
  public void worksWithAwsDynamoDb() {
    AmazonDynamoDB aws = new AmazonDynamoDBClient(
      new BasicAWSCredentials("", "")
    );
    aws.setEndpoint("http://localhost:10500");
    ListTablesResult list = aws.listTables();
    for (String name : list.getTableNames()) {
      System.out.println("table found: " + name);
    }
  }
}

Of course, there won’t be any output because the server starts without any tables. Since the server is empty, you should create tables before every integration test, using createTable() from DynamoDB SDK.

To avoid this type of extra hassle, in the latest version 0.6 of jcabi-dynamodb-maven-plugin we introduced a new goal create-tables:

<plugin>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-dynamodb-maven-plugin</artifactId>
  <executions>
    <execution>
      <goals>
        <goal>create-tables</goal>
      </goals>
      <configuration>
        <tables>
          <table>${basedir}/src/test/dynamodb/foo.json</table>
        </tables>
      </configuration>
    </execution>
  </executions>
</plugin>

The foo.json file used above should contain a JSON request that is sent to DynamoDB Local right after it is up and running. The request should comply with the specification of CreateTable request. For example:

{
  "AttributeDefinitions": [
    {
      "AttributeName": "id",
      "AttributeType": "N"
    }
  ],
  "KeySchema": [
    {
      "AttributeName": "id",
      "KeyType": "HASH"
    }
  ],
  "ProvisionedThroughput": {
    "ReadCapacityUnits": "1",
    "WriteCapacityUnits": "1"
  },
  "TableName": "foo"
}

The table will be created during the pre-integration-test phase and dropped at the post-integration-test phase. Now, we can make our integration test much more meaningful with the help of jcabi-dynamo:

import com.jcabi.dynamo.Attributes;
import com.jcabi.dynamo.Conditions;
import com.jcabi.dynamo.Credentials;
import com.jcabi.dynamo.Region;
import com.jcabi.dynamo.Table;
import org.hamcrest.MatcherAssert;
import org.hamcrest.Matchers;
public class FooITCase {
  @Test
  public void worksWithAwsDynamoDb() {
    Region region = new Region.Simple(new Credentials.Simple("", ""));
    Table table = region.table("foo");
    table.put(
      new Attributes()
        .with("id", 123)
        .with("name", "Robert DeNiro")
    );
    MatcherAssert.assertThat(
      table.frame().where("id", Conditions.equalTo(123)),
      Matchers.notEmpty()
    );
  }
}

The above test will put a new item into the table and then assert that the item is there.

The plugin was tested with three operating systems, and proved to work without problems: Mac OS X 10.8.5, Windows 7 SP1 and Ubuntu Linux 12.04 Desktop.

badge

DynamoDB Local is a locally running copy of Amazon DynamoDB server. Amazon developed the tool and based it on SQLite. It acts as a real DynamoDB service through the RESTful API.

I guess, DynamoDB Local is meant to be used in integration testing and this is how we’re going to use it below.

I use Maven to run all of my Java integration testing using maven-failsafe-plugin. The philosophy of integration testing with Maven is that you start all your supplementary test stubs during the pre-integration-test phase, run your tests in the integration-test phase and then shutdown all stubs during the post-integration-test.

badge

It would be great if it were possible to use DynamoDB Local that way. I didn’t find any Maven plugins for that purpose, so I decided to create my own—jcabi-dynamodb-maven-plugin.

Full usage details for the plugin are explained on its website. However, here is a simple example (get its latest versions in Maven Central):

<plugin>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-dynamodb-maven-plugin</artifactId>
  <executions>
    <execution>
      <goals>
        <goal>start</goal>
        <goal>stop</goal>
      </goals>
      <configuration>
        <port>10500</port>
        <dist>${project.build.directory}/dynamodb-dist</dist>
      </configuration>
    </execution>
  </executions>
</plugin>

The above configuration will start DynamoDB Local right before running integration tests, and then stop it immediately afterwards. The server will listen at TCP port 10500. While the number is used in the example, you’re supposed to use a randomly allocated port instead.

When the DynamoDB Local server is up and running, we can create an integration test for it:

import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.model.ListTablesResult;
public class FooITCase {
  @Test
  public void worksWithAwsDynamoDb() {
    AmazonDynamoDB aws = new AmazonDynamoDBClient(
      new BasicAWSCredentials("", "")
    );
    aws.setEndpoint("http://localhost:10500");
    ListTablesResult list = aws.listTables();
    for (String name : list.getTableNames()) {
      System.out.println("table found: " + name);
    }
  }
}

Of course, there won’t be any output because the server starts without any tables. Since the server is empty, you should create tables before every integration test, using createTable() from DynamoDB SDK.

To avoid this type of extra hassle, in the latest version 0.6 of jcabi-dynamodb-maven-plugin we introduced a new goal create-tables:

<plugin>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-dynamodb-maven-plugin</artifactId>
  <executions>
    <execution>
      <goals>
        <goal>create-tables</goal>
      </goals>
      <configuration>
        <tables>
          <table>${basedir}/src/test/dynamodb/foo.json</table>
        </tables>
      </configuration>
    </execution>
  </executions>
</plugin>

The foo.json file used above should contain a JSON request that is sent to DynamoDB Local right after it is up and running. The request should comply with the specification of CreateTable request. For example:

{
  "AttributeDefinitions": [
    {
      "AttributeName": "id",
      "AttributeType": "N"
    }
  ],
  "KeySchema": [
    {
      "AttributeName": "id",
      "KeyType": "HASH"
    }
  ],
  "ProvisionedThroughput": {
    "ReadCapacityUnits": "1",
    "WriteCapacityUnits": "1"
  },
  "TableName": "foo"
}

The table will be created during the pre-integration-test phase and dropped at the post-integration-test phase. Now, we can make our integration test much more meaningful with the help of jcabi-dynamo:

import com.jcabi.dynamo.Attributes;
import com.jcabi.dynamo.Conditions;
import com.jcabi.dynamo.Credentials;
import com.jcabi.dynamo.Region;
import com.jcabi.dynamo.Table;
import org.hamcrest.MatcherAssert;
import org.hamcrest.Matchers;
public class FooITCase {
  @Test
  public void worksWithAwsDynamoDb() {
    Region region = new Region.Simple(new Credentials.Simple("", ""));
    Table table = region.table("foo");
    table.put(
      new Attributes()
        .with("id", 123)
        .with("name", "Robert DeNiro")
    );
    MatcherAssert.assertThat(
      table.frame().where("id", Conditions.equalTo(123)),
      Matchers.notEmpty()
    );
  }
}

The above test will put a new item into the table and then assert that the item is there.

The plugin was tested with three operating systems, and proved to work without problems: Mac OS X 10.8.5, Windows 7 SP1 and Ubuntu Linux 12.04 Desktop.

© Yegor Bugayenko 2014–2018

W3C Java Validators

QR code

W3C Java Validators

  • comments

badge

A few years ago, I created two Java wrappers for W3C validators: (HTML and CSS). Both wrappers seemed to be working fine and were even listed by W3C on their website in the API section. Until recently, these wrappers have always been part of ReXSL library.

A few days ago, though, I took the wrappers out of ReXSL and published them as a standalone library—jcabi-w3c. Consequently, now seems to be a good time to write a few words about them.

Below is an example that demonstrates how you can validate an HTML document against W3C compliance rules:

import com.jcabi.w3c.ValidatorBuilder;
assert ValidatorBuilder.html()
  .validate("<html>hello, world!</html>")
  .valid();

The valid() method is a black or white indicator that returns false when the document is not valid. Additionally, you can obtain more information through a list of “defects” returned by the W3C server:

Collection<Defect> defects = ValidatorBuilder.html()
  .validate("<html>hello, world!</html>")
  .errors();

The same can be done with CSS:

Collection<Defect> defects = ValidatorBuilder.css()
  .validate("body { font-family: Arial; }")
  .errors();

Personally, I think it is a good practice to validate all of HTML pages produced by your application against W3C during integration testing. It’s not a matter of seeking perfection, but rather of preventing bigger problems later.

These dependencies are mandatory when using jcabi-w3c (get their latest versions in Maven Central):

<dependency>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-w3c</artifactId>
</dependency>
<dependency>
  <groupId>org.glassfish</groupId>
  <artifactId>javax.json</artifactId>
</dependency>
<dependency>
  <groupId>com.sun.jersey</groupId>
  <artifactId>jersey-client</artifactId>
</dependency>
<dependency>
  <groupId>org.hamcrest</groupId>
  <artifactId>hamcrest-core</artifactId>
</dependency>

badge

A few years ago, I created two Java wrappers for W3C validators: (HTML and CSS). Both wrappers seemed to be working fine and were even listed by W3C on their website in the API section. Until recently, these wrappers have always been part of ReXSL library.

A few days ago, though, I took the wrappers out of ReXSL and published them as a standalone library—jcabi-w3c. Consequently, now seems to be a good time to write a few words about them.

Below is an example that demonstrates how you can validate an HTML document against W3C compliance rules:

import com.jcabi.w3c.ValidatorBuilder;
assert ValidatorBuilder.html()
  .validate("<html>hello, world!</html>")
  .valid();

The valid() method is a black or white indicator that returns false when the document is not valid. Additionally, you can obtain more information through a list of “defects” returned by the W3C server:

Collection<Defect> defects = ValidatorBuilder.html()
  .validate("<html>hello, world!</html>")
  .errors();

The same can be done with CSS:

Collection<Defect> defects = ValidatorBuilder.css()
  .validate("body { font-family: Arial; }")
  .errors();

Personally, I think it is a good practice to validate all of HTML pages produced by your application against W3C during integration testing. It’s not a matter of seeking perfection, but rather of preventing bigger problems later.

These dependencies are mandatory when using jcabi-w3c (get their latest versions in Maven Central):

<dependency>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-w3c</artifactId>
</dependency>
<dependency>
  <groupId>org.glassfish</groupId>
  <artifactId>javax.json</artifactId>
</dependency>
<dependency>
  <groupId>com.sun.jersey</groupId>
  <artifactId>jersey-client</artifactId>
</dependency>
<dependency>
  <groupId>org.hamcrest</groupId>
  <artifactId>hamcrest-core</artifactId>
</dependency>

© Yegor Bugayenko 2014–2018

Typical Mistakes in Java Code

QR code

Typical Mistakes in Java Code

  • comments

This page contains most typical mistakes I see in the Java code of people working with me. Static analysis (we’re using qulice can’t catch all of the mistakes for obvious reasons, and that’s why I decided to list them all here.

Let me know if you want to see something else added here, and I’ll be happy to oblige.

All of the listed mistakes are related to object-oriented programming in general and to Java in particular.

Class Names

Your class should be an abstraction of a real life entity with no “validators,” “controllers,” “managers,” etc. If your class name ends with an “-er”—it’s a bad design. BTW, here are my seven virtues of a good object. Also, this post explains this idea in more details: Don’t Create Objects That End With -ER.

And, of course, utility classes are anti-patterns, like StringUtils, FileUtils, and IOUtils from Apache. The above are perfect examples of terrible designs. Read this follow up post: OOP Alternative to Utility Classes

Of course, never add suffixes or prefixes to distinguish between interfaces and classes. For example, all of these names are terribly wrong: IRecord, IfaceEmployee, or RecordInterface. Usually, interface name is the name of a real-life entity, while class name should explain its implementation details. If there is nothing specific to say about an implementation, name it Default, Simple, or something similar. For example:

class SimpleUser implements User {};
class DefaultRecord implements Record {};
class Suffixed implements Name {};
class Validated implements Content {};

Method Names

Methods can either return something or return void. If a method returns something, then its name should explain what it returns, for example (don’t use the get prefix ever):

boolean isValid(String name);
String content();
int ageOf(File file);

If it returns void, then its name should explain what it does. For example:

void save(File file);
void process(Work work);
void append(File file, String line);

You can read more about this idea in Elegant Objects book, section 2.4. There is only one exception to the rule just mentioned—test methods for JUnit. They are explained below.

Test Method Names

Method names in JUnit tests should be created as English sentences without spaces. It’s easier to explain by example:

/**
 * HttpRequest can return its content in Unicode.
 * @throws Exception If test fails
 */
@Test
public void returnsItsContentInUnicode() throws Exception {
}

It’s important to start the first sentence of your Javadoc with the name of the class you’re testing followed by can (or cannot). So, your first sentence should always be similar to “somebody can do something.”

The method name will state exactly the same, but without the subject. If I add a subject at the beginning of the method name, I should get a complete English sentence, as in above example: “HttpRequest returns its content in Unicode.”

Pay attention that the test method doesn’t start with can. Only Javadoc comments use ‘can.’

It’s a good practice to always declare test methods as throwing Exception.

Variable Names

Avoid composite names of variables, like timeOfDay, firstItem, or httpRequest. I mean with both—class variables and in-method ones. A variable name should be long enough to avoid ambiguity in its scope of visibility, but not too long if possible. A name should be a noun in singular or plural form, or an appropriate abbreviation. More about it in this post: A Compound Name Is a Code Smell. For example:

List<String> names;
void sendThroughProxy(File file, Protocol proto);
private File content;
public HttpRequest request;

Sometimes, you may have collisions between constructor parameters and in-class properties if the constructor saves incoming data in an instantiated object. In this case, I recommend to create abbreviations by removing vowels (see how USPS abbreviates street names).

Another example:

public class Message {
  private String recipient;
  public Message(String rcpt) {
    this.recipient = rcpt;
  }
}

In many cases, the best hint for a name of a variable can ascertained by reading its class name. Just write it with a small letter, and you should be good:

File file;
User user;
Branch branch;

However, never do the same for primitive types, like Integer number or String string.

You can also use an adjective, when there are multiple variables with different characteristics. For instance:

String contact(String left, String right);

Constructors

Without exceptions, there should be only one constructor that stores data in object variables. All other constructors should call this one with different arguments. For example:

public class Server {
  private String address;
  public Server(String uri) {
    this.address = uri;
  }
  public Server(URI uri) {
    this(uri.toString());
  }
}

More about it in There Can Be Only One Primary Constructor.

One-time Variables

Avoid one-time variables at all costs. By “one-time” I mean variables that are used only once. Like in this example:

String name = "data.txt";
return new File(name);

This above variable is used only once and the code should be refactored to:

return new File("data.txt");

Sometimes, in very rare cases—mostly because of better formatting—one-time variables may be used. Nevertheless, try to avoid such situations at all costs.

Exceptions

Needless to say, you should never swallow exceptions, but rather let them bubble up as high as possible. Private methods should always let checked exceptions go out.

Never use exceptions for flow control. For example this code is wrong:

int size;
try {
  size = this.fileSize();
} catch (IOException ex) {
  size = 0;
}

Seriously, what if that IOException says “disk is full?” Will you still assume that the size of the file is zero and move on?

Indentation

For indentation, the main rule is that a bracket should either end a line or be closed on the same line (reverse rule applies to a closing bracket). For example, the following is not correct because the first bracket is not closed on the same line and there are symbols after it. The second bracket is also in trouble because there are symbols in front of it and it is not opened on the same line:

final File file = new File(directory,
  "file.txt");

Correct indentation should look like:

StringUtils.join(
  Arrays.asList(
    "first line",
    "second line",
    StringUtils.join(
      Arrays.asList("a", "b")
    )
  ),
  "separator"
);

The second important rule of indentation says that you should put as much as possible on one line - within the limit of 80 characters. The example above is not valid since it can be compacted:

StringUtils.join(
  Arrays.asList(
    "first line", "second line",
    StringUtils.join(Arrays.asList("a", "b"))
  ),
  "separator"
);

Redundant Constants

Class constants should be used when you want to share information between class methods, and this information is a characteristic (!) of your class. Don’t use constants as a replacement of string or numeric literals—very bad practice that leads to code pollution. Constants (as with any object in OOP) should have a meaning in a real world. What meaning do these constants have in the real world:

class Document {
  private static final String D_LETTER = "D"; // bad practice
  private static final String EXTENSION = ".doc"; // good practice
}

Another typical mistake is to use constants in unit tests to avoid duplicate string/numeric literals in test methods. Don’t do this! Every test method should work with its own set of input values.

Use new texts and numbers in every new test method. They are independent. So, why do they have to share the same input constants?

Test Data Coupling

This is an example of data coupling in a test method:

User user = new User("Jeff");
// maybe some other code here
MatcherAssert.assertThat(user.name(), Matchers.equalTo("Jeff"));

On the last line, we couple "Jeff" with the same string literal from the first line. If, a few months later, someone wants to change the value on the third line, he/she has to spend extra time finding where else "Jeff" is used in the same method.

To avoid this data coupling, you should introduce a variable. More about it here: A Few Thoughts on Unit Test Scaffolding.

This page contains most typical mistakes I see in the Java code of people working with me. Static analysis (we’re using qulice can’t catch all of the mistakes for obvious reasons, and that’s why I decided to list them all here.

Let me know if you want to see something else added here, and I’ll be happy to oblige.

All of the listed mistakes are related to object-oriented programming in general and to Java in particular.

Class Names

Your class should be an abstraction of a real life entity with no “validators,” “controllers,” “managers,” etc. If your class name ends with an “-er”—it’s a bad design. BTW, here are my seven virtues of a good object. Also, this post explains this idea in more details: Don’t Create Objects That End With -ER.

And, of course, utility classes are anti-patterns, like StringUtils, FileUtils, and IOUtils from Apache. The above are perfect examples of terrible designs. Read this follow up post: OOP Alternative to Utility Classes

Of course, never add suffixes or prefixes to distinguish between interfaces and classes. For example, all of these names are terribly wrong: IRecord, IfaceEmployee, or RecordInterface. Usually, interface name is the name of a real-life entity, while class name should explain its implementation details. If there is nothing specific to say about an implementation, name it Default, Simple, or something similar. For example:

class SimpleUser implements User {};
class DefaultRecord implements Record {};
class Suffixed implements Name {};
class Validated implements Content {};

Method Names

Methods can either return something or return void. If a method returns something, then its name should explain what it returns, for example (don’t use the get prefix ever):

boolean isValid(String name);
String content();
int ageOf(File file);

If it returns void, then its name should explain what it does. For example:

void save(File file);
void process(Work work);
void append(File file, String line);

You can read more about this idea in Elegant Objects book, section 2.4. There is only one exception to the rule just mentioned—test methods for JUnit. They are explained below.

Test Method Names

Method names in JUnit tests should be created as English sentences without spaces. It’s easier to explain by example:

/**
 * HttpRequest can return its content in Unicode.
 * @throws Exception If test fails
 */
@Test
public void returnsItsContentInUnicode() throws Exception {
}

It’s important to start the first sentence of your Javadoc with the name of the class you’re testing followed by can (or cannot). So, your first sentence should always be similar to “somebody can do something.”

The method name will state exactly the same, but without the subject. If I add a subject at the beginning of the method name, I should get a complete English sentence, as in above example: “HttpRequest returns its content in Unicode.”

Pay attention that the test method doesn’t start with can. Only Javadoc comments use ‘can.’

It’s a good practice to always declare test methods as throwing Exception.

Variable Names

Avoid composite names of variables, like timeOfDay, firstItem, or httpRequest. I mean with both—class variables and in-method ones. A variable name should be long enough to avoid ambiguity in its scope of visibility, but not too long if possible. A name should be a noun in singular or plural form, or an appropriate abbreviation. More about it in this post: A Compound Name Is a Code Smell. For example:

List<String> names;
void sendThroughProxy(File file, Protocol proto);
private File content;
public HttpRequest request;

Sometimes, you may have collisions between constructor parameters and in-class properties if the constructor saves incoming data in an instantiated object. In this case, I recommend to create abbreviations by removing vowels (see how USPS abbreviates street names).

Another example:

public class Message {
  private String recipient;
  public Message(String rcpt) {
    this.recipient = rcpt;
  }
}

In many cases, the best hint for a name of a variable can ascertained by reading its class name. Just write it with a small letter, and you should be good:

File file;
User user;
Branch branch;

However, never do the same for primitive types, like Integer number or String string.

You can also use an adjective, when there are multiple variables with different characteristics. For instance:

String contact(String left, String right);

Constructors

Without exceptions, there should be only one constructor that stores data in object variables. All other constructors should call this one with different arguments. For example:

public class Server {
  private String address;
  public Server(String uri) {
    this.address = uri;
  }
  public Server(URI uri) {
    this(uri.toString());
  }
}

More about it in There Can Be Only One Primary Constructor.

One-time Variables

Avoid one-time variables at all costs. By “one-time” I mean variables that are used only once. Like in this example:

String name = "data.txt";
return new File(name);

This above variable is used only once and the code should be refactored to:

return new File("data.txt");

Sometimes, in very rare cases—mostly because of better formatting—one-time variables may be used. Nevertheless, try to avoid such situations at all costs.

Exceptions

Needless to say, you should never swallow exceptions, but rather let them bubble up as high as possible. Private methods should always let checked exceptions go out.

Never use exceptions for flow control. For example this code is wrong:

int size;
try {
  size = this.fileSize();
} catch (IOException ex) {
  size = 0;
}

Seriously, what if that IOException says “disk is full?” Will you still assume that the size of the file is zero and move on?

Indentation

For indentation, the main rule is that a bracket should either end a line or be closed on the same line (reverse rule applies to a closing bracket). For example, the following is not correct because the first bracket is not closed on the same line and there are symbols after it. The second bracket is also in trouble because there are symbols in front of it and it is not opened on the same line:

final File file = new File(directory,
  "file.txt");

Correct indentation should look like:

StringUtils.join(
  Arrays.asList(
    "first line",
    "second line",
    StringUtils.join(
      Arrays.asList("a", "b")
    )
  ),
  "separator"
);

The second important rule of indentation says that you should put as much as possible on one line - within the limit of 80 characters. The example above is not valid since it can be compacted:

StringUtils.join(
  Arrays.asList(
    "first line", "second line",
    StringUtils.join(Arrays.asList("a", "b"))
  ),
  "separator"
);

Redundant Constants

Class constants should be used when you want to share information between class methods, and this information is a characteristic (!) of your class. Don’t use constants as a replacement of string or numeric literals—very bad practice that leads to code pollution. Constants (as with any object in OOP) should have a meaning in a real world. What meaning do these constants have in the real world:

class Document {
  private static final String D_LETTER = "D"; // bad practice
  private static final String EXTENSION = ".doc"; // good practice
}

Another typical mistake is to use constants in unit tests to avoid duplicate string/numeric literals in test methods. Don’t do this! Every test method should work with its own set of input values.

Use new texts and numbers in every new test method. They are independent. So, why do they have to share the same input constants?

Test Data Coupling

This is an example of data coupling in a test method:

User user = new User("Jeff");
// maybe some other code here
MatcherAssert.assertThat(user.name(), Matchers.equalTo("Jeff"));

On the last line, we couple "Jeff" with the same string literal from the first line. If, a few months later, someone wants to change the value on the third line, he/she has to spend extra time finding where else "Jeff" is used in the same method.

To avoid this data coupling, you should introduce a variable. More about it here: A Few Thoughts on Unit Test Scaffolding.

© Yegor Bugayenko 2014–2018

Java XML Parsing Made Easy

QR code

Java XML Parsing Made Easy

  • comments

badge

Unlike with many other modern languages, parsing XML in Java requires more than one line of code. XML traversing using XPath takes even more code, and I find this is unfair and annoying.

I’m a big fan of XML and use it it in almost every Java application. Some time ago, I decided to put all of that XML-to-DOM parsing code into a small library—jcabi-xml.

Put simply, the library is a convenient wrapper for JDK-native DOM manipulations. That’s why it is small and dependency-free. With the following example, you can see just how simple XML parsing can be:

import com.jcabi.xml.XML;
import com.jcabi.xml.XMLDocument;
XML xml = new XMLDocument(
  "<root><a>hello</a><b>world!</b></root>"
);

Now, we have an object of interface XML that can traverse the XML tree and convert it back to text.

For example:

// outputs "hello"
System.out.println(xml.xpath("/root/a/text()").get(0));
// outputs the entire XML document
System.out.println(xml.toString());

Method xpath() allows you to find a collection of text nodes or attributes in the document, and then convert them to a collection of strings, using XPath query:

// outputs "hello" and "world"
for (String text : xml.xpath("/root/*/text()")) {
  System.out.println(text);
}

Method nodes() enables the same XPath search operation, but instead returns a collection of instances of XML interface:

// outputs "<a>hello</a>" and "<b>world</b>"
for (XML node : xml.nodes("/root/*"))
  System.out.println(node);
}

Besides XML parsing, printing and XPath traversing, jcabi-xml also provides XSD validation and XSL transformations. I’ll write about those features in the next post :)

PS. Also, check this: XML/XPath Matchers for Hamcrest.

badge

Unlike with many other modern languages, parsing XML in Java requires more than one line of code. XML traversing using XPath takes even more code, and I find this is unfair and annoying.

I’m a big fan of XML and use it it in almost every Java application. Some time ago, I decided to put all of that XML-to-DOM parsing code into a small library—jcabi-xml.

Put simply, the library is a convenient wrapper for JDK-native DOM manipulations. That’s why it is small and dependency-free. With the following example, you can see just how simple XML parsing can be:

import com.jcabi.xml.XML;
import com.jcabi.xml.XMLDocument;
XML xml = new XMLDocument(
  "<root><a>hello</a><b>world!</b></root>"
);

Now, we have an object of interface XML that can traverse the XML tree and convert it back to text.

For example:

// outputs "hello"
System.out.println(xml.xpath("/root/a/text()").get(0));
// outputs the entire XML document
System.out.println(xml.toString());

Method xpath() allows you to find a collection of text nodes or attributes in the document, and then convert them to a collection of strings, using XPath query:

// outputs "hello" and "world"
for (String text : xml.xpath("/root/*/text()")) {
  System.out.println(text);
}

Method nodes() enables the same XPath search operation, but instead returns a collection of instances of XML interface:

// outputs "<a>hello</a>" and "<b>world</b>"
for (XML node : xml.nodes("/root/*"))
  System.out.println(node);
}

Besides XML parsing, printing and XPath traversing, jcabi-xml also provides XSD validation and XSL transformations. I’ll write about those features in the next post :)

PS. Also, check this: XML/XPath Matchers for Hamcrest.

© Yegor Bugayenko 2014–2018

Object-Oriented DynamoDB API

QR code

Object-Oriented DynamoDB API

  • comments

badge

I’m a big fan of cloud computing in general and of Amazon Web Services in particular. I honestly believe that in a few years big providers will host all, or almost all, computing and storage resources. When this is the case, we won’t have to worry too much anymore about downtime, backups and system administrators. DynamoDB is one of the steps towards this future.

DynamoDB is a NoSQL database accessible through RESTful JSON API. Its design is relatively simple. There are tables, which basically are collections of data structures, or in AWS terminology, “items.”

Every item has a mandatory “hash,” an optional “range” and a number of other optional attributes. For instance, take the example table depts:

+------+--------+---------------------------+
| dept | worker | Attributes                |
+------+--------+---------------------------+
| 205  | Jeff   | job="manager", sex="male" |
| 205  | Bob    | age=43, city="Chicago"    |
| 398  | Alice  | age=27, job="architect"   |
+------+--------+---------------------------+

For Java, Amazon provides an SDK, which mirrors all RESTful calls to Java methods. The SDK works fine, but is designed in a pure procedural style.

Let’s say we want to add a new item to the table above. RESTful call putItem looks like (in essence):

putItem:
  tableName: depts
  item:
    dept: 435
    worker: "William"
    job: "programmer"

This is what the Amazon server needs to know in order to create a new item in the table. This is how you’re supposed to make this call through the AWS Java SDK:

PutItemRequest request = new PutItemRequest();
request.setTableName("depts");
Map<String, AttributeValue> attributes = new HashMap<>();
attributes.put("dept", new AttributeValue(435));
attributes.put("worker", new AttributeValue("William"));
attributes.put("job", new AttributeValue("programmer));
request.setItem(attributes);
AmazonDynamoDB aws = // instantiate it with credentials
try {
  aws.putItem(request);
} finally {
  aws.shutdown();
}

The above script works fine, but there is one major drawback—it is not object oriented. It is a perfect example of an imperative procedural programming.

To allow you to compare, let me show what I’ve done with jcabi-dynamo. Here is my code, which does exactly the same thing, but in an object-oriented way:

Region region = // instantiate it with credentials
Table table = region.table("depts");
Item item = table.put(
  new Attributes()
    .with("dept", 435)
    .with("worker", "William")
    .with("job", "programmer")
);

My code is not only shorter, but it also employs encapsulation and separates responsibilities of classes. Table class (actually it is an interface internally implemented by a class) encapsulates information about the table, while Item encapsulates item details.

We can pass an item as an argument to another method and all DynamoDB related implementation details will be hidden from it. For example, somewhere later in the code:

void sayHello(Item item) {
  System.out.println("Hello, " + item.get("worker"));
}

In this script, we don’t know anything about DynamoDB or how to deal with its RESTful API. We interact solely with an instance of Item class.

By the way, all public entities in jcabi-dynamo are Java interfaces. Thanks to that, you can test and mock the library completely (but I would recommend to use DynamoDB Local and create integration tests).

Let’s consider a more complex example, which would take a page of code if we were to use a bare AWS SDK. Let’s say that we want to remove all workers from our table who work as architects:

Region region = // instantiate it with credentials
Iterator<Item> workers = region.table("depts").frame()
  .where("job", Condition.equalTo("architect"));
while (workers.hasNext()) {
  workers.remove();
}

jcabi-dynamo has saved a lot of code lines in a few of my projects. You can see it in action at rultor-users.

The library ships as a JAR dependency in Maven Central (get its latest versions from Maven Central):

<dependency>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-dynamo</artifactId>
</dependency>

badge

I’m a big fan of cloud computing in general and of Amazon Web Services in particular. I honestly believe that in a few years big providers will host all, or almost all, computing and storage resources. When this is the case, we won’t have to worry too much anymore about downtime, backups and system administrators. DynamoDB is one of the steps towards this future.

DynamoDB is a NoSQL database accessible through RESTful JSON API. Its design is relatively simple. There are tables, which basically are collections of data structures, or in AWS terminology, “items.”

Every item has a mandatory “hash,” an optional “range” and a number of other optional attributes. For instance, take the example table depts:

+------+--------+---------------------------+
| dept | worker | Attributes                |
+------+--------+---------------------------+
| 205  | Jeff   | job="manager", sex="male" |
| 205  | Bob    | age=43, city="Chicago"    |
| 398  | Alice  | age=27, job="architect"   |
+------+--------+---------------------------+

For Java, Amazon provides an SDK, which mirrors all RESTful calls to Java methods. The SDK works fine, but is designed in a pure procedural style.

Let’s say we want to add a new item to the table above. RESTful call putItem looks like (in essence):

putItem:
  tableName: depts
  item:
    dept: 435
    worker: "William"
    job: "programmer"

This is what the Amazon server needs to know in order to create a new item in the table. This is how you’re supposed to make this call through the AWS Java SDK:

PutItemRequest request = new PutItemRequest();
request.setTableName("depts");
Map<String, AttributeValue> attributes = new HashMap<>();
attributes.put("dept", new AttributeValue(435));
attributes.put("worker", new AttributeValue("William"));
attributes.put("job", new AttributeValue("programmer));
request.setItem(attributes);
AmazonDynamoDB aws = // instantiate it with credentials
try {
  aws.putItem(request);
} finally {
  aws.shutdown();
}

The above script works fine, but there is one major drawback—it is not object oriented. It is a perfect example of an imperative procedural programming.

To allow you to compare, let me show what I’ve done with jcabi-dynamo. Here is my code, which does exactly the same thing, but in an object-oriented way:

Region region = // instantiate it with credentials
Table table = region.table("depts");
Item item = table.put(
  new Attributes()
    .with("dept", 435)
    .with("worker", "William")
    .with("job", "programmer")
);

My code is not only shorter, but it also employs encapsulation and separates responsibilities of classes. Table class (actually it is an interface internally implemented by a class) encapsulates information about the table, while Item encapsulates item details.

We can pass an item as an argument to another method and all DynamoDB related implementation details will be hidden from it. For example, somewhere later in the code:

void sayHello(Item item) {
  System.out.println("Hello, " + item.get("worker"));
}

In this script, we don’t know anything about DynamoDB or how to deal with its RESTful API. We interact solely with an instance of Item class.

By the way, all public entities in jcabi-dynamo are Java interfaces. Thanks to that, you can test and mock the library completely (but I would recommend to use DynamoDB Local and create integration tests).

Let’s consider a more complex example, which would take a page of code if we were to use a bare AWS SDK. Let’s say that we want to remove all workers from our table who work as architects:

Region region = // instantiate it with credentials
Iterator<Item> workers = region.table("depts").frame()
  .where("job", Condition.equalTo("architect"));
while (workers.hasNext()) {
  workers.remove();
}

jcabi-dynamo has saved a lot of code lines in a few of my projects. You can see it in action at rultor-users.

The library ships as a JAR dependency in Maven Central (get its latest versions from Maven Central):

<dependency>
  <groupId>com.jcabi</groupId>
  <artifactId>jcabi-dynamo</artifactId>
</dependency>

© Yegor Bugayenko 2014–2018

Fluent Java HTTP Client

QR code

Fluent Java HTTP Client

  • comments

badge

In the world of Java, there are plenty of HTTP clients from which to choose. Nevertheless, I decided to create a new one because none of the other clients satisfied fully all of my requirements. Maybe, I’m too demanding. Still, this is how my jcabi-http client interacts when you make an HTTP request and expect a successful HTML page in return:

String html = new JdkRequest("https://www.google.com")
  .uri().path("/users").queryParam("id", 333).back()
  .method(Request.GET)
  .header("Accept", "text/html")
  .fetch()
  .as(RestResponse.class)
  .assertStatus(HttpURLConnection.HTTP_OK)
  .body();

I designed this new client with the following requirements in mind:

Simplicity

For me, this was the most important requirement. The client must be simple and easy to use. In most cases, I need only to make an HTTP request and parse the JSON response to return a value. For example, this is how I use the new client to return a current EUR rate:

String uri = "http://www.getexchangerates.com/api/latest.json";
String rate = new JdkRequest(uri)
  .header("Accept", "application/json")
  .fetch()
  .as(JsonResponse.class)
  .json().readArray().getJsonObject(0)
  .getString("EUR");

I assume that the above is easy to understand and maintain.

Fluent Interface

The new client has to be fluent, which means that the entire server interaction fits into one Java statement. Why is this important? I think that fluent interface is the most compact and expressive way to perform multiple imperative calls. To my knowledge, none of the existing libraries enable this type of fluency.

Testable and Extensible

I’m a big fan of interfaces, mostly because they make your designs both cleaner and highly extensible at the same time. In jcabi-http, there are five interfaces extended by 20 classes.

Request is an interface, as well as Response, RequestURI, and RequestBody exposed by it.

Use of interfaces makes the library highly extensible. For example, we have JdkRequest and ApacheRequest, which make actual HTTP calls to the server using two completely different technologies: (JDK HttpURLConnection and Apache HTTP Client, respectively). In the future, it will be possible to introduce new implementations without breaking existing code.

Say, for instance, I want to fetch a page and then do something with it. These two calls perform the task differently, but the end results are the same:

String uri = "http://www.google.com";
Response page;
page = new JdkRequest(uri).fetch();
page = new ApacheRequest(uri).fetch();

XML and JSON Out-of-the-Box

There are two common standards that I wanted the library to support right out of the box. In most cases, the response retrieved from a server is in either XML or JSON format. It has always been a hassle, and extra work, for me to parse the output to take care of formatting issues.

jcabi-http client supports them both out of the box, and it’s possible to add more formats in the future as needed. For example, you can fetch XML and retrieve a string value from its element:

String name = new JdkRequest("http://my-api.example.com")
  .header("Accept", "text/xml")
  .fetch()
  .as(XmlResponse.class)
  .xml().xpath("/root/name/text()").get(0);

Basically, the response produced by fetch() is decorated by XmlResponse. This then exposes the xml() method that returns an instance of the XML interface.

The same can be done with JSON through the Java JSON API (JSR-353).

None of the libraries that I’m aware of or worked with offer this feature.

Immutable

The last requirement, but certainly not the least important, is that I need all interfaces of the library to be annotated with @Immutable. This is important because I need to be able to encapsulate an instance of Request in other immutable classes.

ps. A short summary of this article was published at JavaLobby

badge

In the world of Java, there are plenty of HTTP clients from which to choose. Nevertheless, I decided to create a new one because none of the other clients satisfied fully all of my requirements. Maybe, I’m too demanding. Still, this is how my jcabi-http client interacts when you make an HTTP request and expect a successful HTML page in return:

String html = new JdkRequest("https://www.google.com")
  .uri().path("/users").queryParam("id", 333).back()
  .method(Request.GET)
  .header("Accept", "text/html")
  .fetch()
  .as(RestResponse.class)
  .assertStatus(HttpURLConnection.HTTP_OK)
  .body();

I designed this new client with the following requirements in mind:

Simplicity

For me, this was the most important requirement. The client must be simple and easy to use. In most cases, I need only to make an HTTP request and parse the JSON response to return a value. For example, this is how I use the new client to return a current EUR rate:

String uri = "http://www.getexchangerates.com/api/latest.json";
String rate = new JdkRequest(uri)
  .header("Accept", "application/json")
  .fetch()
  .as(JsonResponse.class)
  .json().readArray().getJsonObject(0)
  .getString("EUR");

I assume that the above is easy to understand and maintain.

Fluent Interface

The new client has to be fluent, which means that the entire server interaction fits into one Java statement. Why is this important? I think that fluent interface is the most compact and expressive way to perform multiple imperative calls. To my knowledge, none of the existing libraries enable this type of fluency.

Testable and Extensible

I’m a big fan of interfaces, mostly because they make your designs both cleaner and highly extensible at the same time. In jcabi-http, there are five interfaces extended by 20 classes.

Request is an interface, as well as Response, RequestURI, and RequestBody exposed by it.

Use of interfaces makes the library highly extensible. For example, we have JdkRequest and ApacheRequest, which make actual HTTP calls to the server using two completely different technologies: (JDK HttpURLConnection and Apache HTTP Client, respectively). In the future, it will be possible to introduce new implementations without breaking existing code.

Say, for instance, I want to fetch a page and then do something with it. These two calls perform the task differently, but the end results are the same:

String uri = "http://www.google.com";
Response page;
page = new JdkRequest(uri).fetch();
page = new ApacheRequest(uri).fetch();

XML and JSON Out-of-the-Box

There are two common standards that I wanted the library to support right out of the box. In most cases, the response retrieved from a server is in either XML or JSON format. It has always been a hassle, and extra work, for me to parse the output to take care of formatting issues.

jcabi-http client supports them both out of the box, and it’s possible to add more formats in the future as needed. For example, you can fetch XML and retrieve a string value from its element:

String name = new JdkRequest("http://my-api.example.com")
  .header("Accept", "text/xml")
  .fetch()
  .as(XmlResponse.class)
  .xml().xpath("/root/name/text()").get(0);

Basically, the response produced by fetch() is decorated by XmlResponse. This then exposes the xml() method that returns an instance of the XML interface.

The same can be done with JSON through the Java JSON API (JSR-353).

None of the libraries that I’m aware of or worked with offer this feature.

Immutable

The last requirement, but certainly not the least important, is that I need all interfaces of the library to be annotated with @Immutable. This is important because I need to be able to encapsulate an instance of Request in other immutable classes.

ps. A short summary of this article was published at JavaLobby

© Yegor Bugayenko 2014–2018

sixnines availability badge